NIST Computer Guide to IPsec VPNs

IPsec is a framework of open standards for ensuring private communications over public networks. It has become the most common network layer security control, typically used to create a virtual private network (VPN). A VPN is a virtual network built on top of existing physical networks that can provide a secure communications mechanism for data and control information transmitted between networks. VPNs are used most often to protect communications carried over public networks such as the Internet. A VPN can provide several types of data protection, including confidentiality, integrity, data origin authentication, replay protection and access control. Although VPNs can reduce the risks of networking, they cannot totally eliminate them. For example, a VPN implementation may have flaws in algorithms or software, or a VPN may be set up with insecure configuration settings and values. Both of these flaws can be exploited by attackers. There are three primary models for VPN architectures, as follows:

Gateway-to-gateway. This model protects communications between two specific networks, such as an organization’s main office network and a branch office network, or two business partners’ networks.

Host-to-gateway. This model protects communications between one or more individual hosts and a specific network belonging to an organization. The host-to-gateway model is most often used to allow hosts on unsecured networks, such as traveling employees and telecommuters, to gain access to internal organizational services, such as the organization’s e-mail and Web servers.

Host-to-host. A host-to-host architecture protects communication between two specific computers. It is most often used when a small number of users need to use or administer a remote system that requires the use of inherently insecure protocols.

The guide provides an overview of the types of security controls that can provide protection for Transmission Control Protocol/Internet Protocol (TCP/IP) network communications, which are widely used throughout the world. TCP/IP communications are composed of four layers that work together: application, transport, network, and data link. Security controls exist for network communications at each of the four layers. As data is prepared for transport, it is passed from the highest to the lowest layer, with each layer adding more information. Because of this, a security control at a higher layer cannot provide full protection for lower layers, because the lower layers add information to the communications after the higher layer security controls have been applied. The primary disadvantage of lower layer security controls is that they are less flexible and granular than higher layer controls. Accordingly, network layer controls have become widely used for securing communications because they provide a more balanced solution than the highest layer and lowest layer security controls.

IPsec is a network layer security protocol with the following components:

Two security protocols, Authentication Header (AH) and Encapsulating Security Payload (ESP). AH can provide integrity protection for packet headers and data, but it cannot encrypt them. ESP can provide encryption and integrity protection for packets, but it cannot protect the outermost IP header, as AH can. However, this protection is not needed in most cases. Accordingly, ESP is used much more frequently than AH because of its encryption capabilities, as well as other operational advantages which will be described in this document. For a VPN, which requires confidential communications, ESP is the natural choice.

Internet Key Exchange (IKE) protocol. IPsec uses IKE to negotiate IPsec connection settings; authenticate endpoints to each other; define the security parameters of IPsec-protected connections; negotiate secret keys; and manage, update, and delete IPsec-protected communication channels.