Using IPSec (IP Security Protocol)


Table of Contents

Portions of this document were taken from:

13.1 - What is IPSec?

IPSec is a set of extensions to the IP protocol family. It provides cryptographic security services. These services allow for authentication, integrity, access control, and confidentiality. IPSec provides similar services as SSL, but at the network layer, in a way that is completely transparent to your applications, and much more powerful. We say this because your applications do not have to have any knowledge of IPSec to be able to use it. You can use any IP protocol over IPSec. You can create encrypted tunnels (VPNs), or just do encryption between computers. Since you have so many options, IPSec is rather complex (much more so then SSL!)

Before you start using IPSec, we strongly recommend that you check out the "recommended reading" of part 6 of the FAQ. In particular, if you don't already understand it, the Understanding IP Addressing document is highly recommended.

In a logical sense, IPSec works in any of these three ways:

In every scenario that involves a network, we mean to imply router. As in, Host-to-Router (and this router controls and encrypts traffic for a particular Network.)

As you can see, IPSec can be used to tunnel traffic for VPN connections. However, its utility reaches beyond VPNs. With a central Internet Key Exchange registry, every machine on the internet could talk to another one and employ powerful encryption and authentication!

13.2 - That's nice, but why do I want to use IPSec?

The internet protocol, IP, aka IPv4, does not inherently provide any protection to your transferred data. It does not even guarantee that the sender is who he says he is. IPSec tries to remedy this. These services are considered distinct, but the IPSec supports them in a uniform manner.



Make sure it is hard for anyone but the receiver to understand what data has been communicated. You do not want anyone to see your passwords when logging into a remote machine over the Internet.



Guarantee that the data does not get changed on the way. If you are on a line carrying invoicing data you probably want to know that the amounts and account numbers are correct and not altered while in-transit.



Sign your data so that others can see that it is really you that sent it. It is clearly nice to know that documents are not forged.


Replay protection

We need ways to ensure a transaction can only be carried out once unless we are authorized to repeat it. I.e. it should not be possible for someone to record a transaction, and then replaying it verbatim, in order to get an effect of multiple transactions being received by the peer. Consider the attacker has got to know what the traffic is all about by other means than cracking the encryption, and that the traffic causes events favourable for him, like depositing money into his account. We need to make sure he cannot just replay that traffic later. WARNING: as per the standards specification, replay protection is not performed when using manual-keyed IPsec (e.g., when using ipsecadm(8)).

13.3 - What are the protocols behind IPSec?

IPSec provides confidentiality, integrity, authenticity, and replay protection through two new protocols. These protocols are called AH, Authentication header, and ESP, Encapsulated security payload.

AH provides authentication, integrity, and replay protection (but not confidentiality). Its main difference with ESP is that AH also secures parts of the IP header of the packet (like the source/destination addresses).

ESP can provide authentication, integrity, replay protection, and confidentiality of the data (it secures everything in the packet that follows the header). Replay protection requires authentication and integrity (these two go always together). Confidentiality (encryption) can be used with or without authentication/integrity. Similarly, one could use authentication/integrity with or without confidentiality.

13.4 - On the wire format

The Authentication Header (AH) comes after the basic IP header and contains cryptographic hashes of the data and identification information. The hashes can also cover the invariant parts of the IP header itself. There are several different RFCs giving a choice of actual algorithms to use in the AH, however they all must follow the guidelines specified in RFC2402.

The Encapsulating Security Payload (ESP) header, allows for rewriting of the payload in encrypted form. The ESP header does not consider the fields of the IP header before it and therefore makes no guarantees about anything except the payload. The various types of ESP applicable must follow RFC2406. An ESP header can also provide authentication for the payload, (but not the outer header).

An orthogonal (mostly) division of IPSec functionality is applied depending on whether the endpoint doing the IPSec encapsulation is the original source of the data or a gateway:

IPSec secured links are defined in terms of Security Associations (SAs). Each SA is defined for a single unidirectional flow of data, and usually (ignoring multicast) from one single point to another, covering traffic distinguishable by some unique selector. All traffic flowing over a single SA is treated the same. Some traffic may be subject to several SAs, each of which applies some transform. Groups of SAs are called an SA Bundle. Incoming packets can be assigned to a particular SA by the three defining fields, (Destination IP address, Security Parameter Index, security protocol). SPI can be considered a cookie that is handed out by the receiver of the SA when the parameters of the connection are negotiated. The security protocol must be either AH or ESP. Since the IP address of the receiver is part of the triple, this is a guaranteed unique value. They can be found from the outer IP header and the first security header (which contains the SPI and the security protocol).

An example of a tunnel mode AH packet is:

IPhdr AH IPhdr2 TCPhdr data

An example of a transport mode AH packet is:

IPhdr AH TCPhdr data

Because an ESP header cannot authenticate the outer IP header, it is useful to combine an AH and an ESP header to get the following:

IPhdr AH ESP TCPhdr data

This is called Transport Adjacency. The tunneling version would look like:

IPhdr AH ESP IPhdr2 TCPhdr data

However it is not specifically mentioned in the RFC. As with Transport adjacency, this would authenticate the entire packet except a few headers in the IP header and also encrypt the payload (seen in italics). When an AH and an ESP header are directly applied together like this, the order of the headers should be as shown. It is possible in tunnel mode, to do arbitrary recursive encapsulation so that order is not specified.

13.5 - Configuring IPSec

How the IPSec systems and gateways are configured is to some extent left to the designer, however the RFC has some strong recommendations as to how this should be implemented, so as to minimize confusion.

There are two administrative entities that control what happens to a packet. One is the Security Association Database (SAD, referred to as TDB or TDB table throughout OpenBSD's IPSec source code) and the other is the Security Policy Database (SPD).

They are similar in that given a number of selectors that describe some traffic, they will deliver an entry that describes the processing needed. However, the SPD is two steps removed from the actual processing: the SPD is used for outgoing packets, to decide what SAD entries should be used, and the SAD entries in turn describe the actual process and the parameters for it. The SPD entries specify the existing SAD entries to use (if it's a bundle there can be more than 1), but if there is not already a suitable one, it is used to create new ones. The fields of the SA being created can be taken either from the SPD entry or from the packet that initiated the creation.

Outgoing packets go from the SPD entry to the specific SA, to get encoding parameters. Incoming packets get to the correct SA directly using the SPI/DestIP/Proto triple, and from there get to the SPD entry.

The SPD can also specify what traffic should bypass IPSec and what should be dropped, so it must also be consulted for incoming non-IPSec traffic. SPD entries must be explicitly ordered as several might match a particular packet, and the processing must be reproducible.

The SPD can be thought of as similar to a packet filter where the actions decided upon are the activation of SA processes. Selectors can include src and dest address, port numbers if relevant, application and user IDs if available (only on host based transport SAs), hostnames, security sensitivity levels, protocols, etc.

A SAD entry would include:

A SPD entry would contain:

Each SA can define one ESP header and one AH header. An IPSec session must have one or the other or both, but cannot be defined with neither - otherwise there would be no headers to specify the SPI to look up the SA. The RFC doesn't say what would happen if the AH and ESP headers disagree about the SPI value. One would presume this would imply multiple SAs in a bundle.

The SPD in OpenBSD is managed through the ipsecadm flow command. (You would only make changes to it if you were using manual keying.) SAD entries can be set manually with ipsecadm(8), however the IETF has also defined automatic mechanisms for initialization of sessions and such things as key exchange. OpenBSD implements both Photuris (RFC2522, and RFC2523) and ISAKMP automatic key exchange (RFC2407, RFC2408, and RFC2409) in the photurisd(8) and isakmpd(8) daemons.

13.6 - How do I setup IPSec with manual keying?

Manual keying is the easiest way to get started with IPSec. You can setup encryption between networks, to create VPNs using this method. After you have read this section, you may want to investigate using /usr/share/ipsec/rc.vpn to set this up for you automatically.

First, you need to turn on IP AH and IP ESP options in the OpenBSD kernel (if you are only using ESP, such as with the rc.vpn script, or as with the example below, then you do not need to enable AH. In fact, there may even be security concerns related to enabling AH if you are not using it).

There is a nice sysctl to enable these protocols.

You can edit /etc/sysctl.conf to turn these on at boot time. You need to remove the # mark from in front of net.inet.esp.enable and/or net.inet.ah.enable (depending on which you plan to use) and make sure they are set to 1.

You also need to generate your manual keys. Since the security of the VPN is based on these keys being unguessable, it is very important that the keys be chosen using a strong random source. One practical method of generating them is by using the random(4) device. To produce 160 bits of randomness, for example, do:

The number of bits produced is important. Different cipher types may require different sized keys.
Cipher    Key Length
DES       56 bits
3DES      168 bits
BLF       Variable (40-160, 160 bits recommended)
CAST      Variable (40-128, 128 bits recommended)
SKIPJACK  80 bits

Now, you need to setup SAs, or Security Associations. A Security Association is a combination of your IP addresses, an SPI, and your security protocol (AH and/or ESP). The IP addresses are both your own and that of your destination. The SPI, or Security Parameter Index, is a number that OpenBSD uses to classify different SAs.

These examples only use ESP to encrypt your traffic. ESP includes authentication of the contained encrypted data, but does not authenticate the surrounding IP header, as AH would. This "limited authentication" is nevertheless quite sufficient in most cases, especially for ESP in a tunnel environment.


Let's put this into practice with two routers, and

On Host

On Host

Notice that the SPIs are different. See On the wire format for a complete description of what the SPI is and where it is used.

Now that you have your Security Associations in place, set up your flows.


So, right here, two flows will be created, one the local source address, which covers all packets originating from the local host to the destination, as well as a flow from the destination back to the local host.


If you want less overhead on your Host-to-Host VPNs, creating the SPI without -forcetunnel will let you use transport mode (whereas, -forcetunnel makes sure all of the IP packet, including the IP header, are encapsulated by SPI). If either the source or destination is a network, you will have to use tunnel mode. Creating an SA to and/or from a network will automatically ensure tunnel mode SPIs are being created.

This is a simple way to start using IPSec.

You can use IPSec to tunnel private IP address spaces over the Internet. Here is a good example... We want to tunnel, which is behind, to and which are behind These examples were generated using the rc.vpn script.

As you can see, when you are using manual keying with IPSec, you have to specify exactly what you want done. It won't guess for you. Look at these examples...


First, set up the security associations (SAs):
(This sets up the SPIs, encryption methods, and keys.) Next, setup a flow from to Next, setup a flow from, which is behind, to Next, setup a flow from, which is behind, to Now, setup a flow from, which is behind to the router OK, setup a flow from, which is behind, to the router Finally, setup a flow from the router to



Same as before, we setup the SAs... Now, this is the reverse side... Setup a flow from the router to Setup a flow from the network, which is behind, to Setup a flow from the network, which is behind, to Now, setup a flow from, which is behind, to the router We're almost done... Two flows left to get and from the router to the router

If you have been using ipsecadm, and you want to get rid of any work that you've done, and start from scratch, do

This will flush all IPSec info (SPIs, flows, routing entries) from your system.

13.7 - How do I setup photurisd?

Photuris is not widely used and is still considered experimental as far as the RFC status is concerned. However, many people have used it with OpenBSD.

To setup your photurisd, first edit /etc/photuris/secrets.conf on each host with photurisd.

Change "This should be changed." to a key of your choice on the local config, and another key of your choice on the remote config. (Use the same config on your remote box but swap "local" and "remote" so that it sees itself as the local key.) Note that these keys will be replaced in a future version of photurisd that will carry out its initial key exchange with public keys.

Make sure net.inet.ah.enable is set to 1.

And run startkey. Now, run tcpdump to verify that your packets are being encrypted with AH. (Run a ping in another window or session to generate traffic.) You can also try using tcpdump by the host address if you aren't getting anything. You can make photurisd automatically set the source and destination host or networks in /etc/photuris/photuris.startup

13.8 - How do I setup isakmpd?

If you are thinking about VPNs or other traditional applications of IPSec, you probably are going to use ISAKMP. Some commercial implementations of IPSec do not provide any manual keying ability, instead they require you to use some form of ISAKMP.


13.8.1 - What is isakmpd?

ISAKMP (sometimes referred to as IKE, or Internet Key Exchange) is the key exchange mechanism for the VPN. It meets security concerns using the methods mentioned in RFC 2407, RFC 2408 and RFC 2409. ISAKMP manages the exchange of cryptographic keys that you would normally have to manage with ipsecadm(8). It employs a two-phase process for establishing the IPSec parameters between two IPSec nodes.

Phase 1 - The two ISAKMP peers establish a secure, authenticated channel upon which to communicate between two daemons. This establishes a Security Association (SA) between both hosts. Main Mode and Aggressive Mode are the methods used to establish this channel. Main Mode sends the various authentication information in a certain sequence, providing identity protection. Aggressive Mode does not provide identity protection because all of the authentication information is sent at the same time. Aggressive mode should only be used in such cases where network bandwidth is of concern.

Phase 2 - Security Associations are negotiated on behalf of IPSec. Phase 2 is establishes tunnels or endpoint SAs between IPSec hosts. Quick Mode is used in Phase 2 because there is not need to repeat a full authentication because Phase 1 has already established the SAs.

In brief, Phase 1 is used to get a secure channel in which to do the (quicker) phase 2 setups. There can be multiple phase 2 setups within the same phase 1 channel. Phase 2 is used to setup the actual tunnels. In Phase 1, your IPSec nodes establish a connection where they exchange authentication (Either a X509 certificate or a pre-shared secret). This allows each end to make sure the other end is authenticated. Phase 2 is an exchange of keys to determine how the data between the two will be encrypted.

13.8.2 - How do I get started with isakmpd?

By default, OpenBSD comes with the proper binaries for ISAKMP and the IPSec stack. Unfortunately, the same cannot be said for the sample files. To retrieve them, you need to grab: /usr/src/sbin/isakmpd/samples/VPN-east.conf and /usr/src/sbin/isakmpd/samples/policy from the source tree. You can either use your CD (if you have one), cvsweb, or the command line CVS client. (Cvsweb has VPN-east.conf and policy readily available). For the purposes of this example, copy policy to /etc/isakmpd/isakmpd.policy. Copy VPN-east.conf to /etc/isakmpd/isakmpd.conf. Here we attempt to show you how to setup a VPN (tunnel). If you want to use isakmpd between single hosts, there are other configuration files in the samples directory. The manual pages have detailed information.. Don't forget isakmpd.conf(5) and isakmpd.policy(5).

Your first step is to turn on esp. The top of section 13.6 tells you how to do this both for run-time and boot. Next, you need to edit /etc/isakmpd.policy. This file tells ISAKMP who can access IPSec. In this scenario, the policy file states that anybody who sends data using Encapsulate Security Payload(ESP), and has authenticated with the passphrase mekmitasdigoat (or whatever passphrase you determine), is allowed to communicate with isakmpd. You can modify this file to let ISAKMP know that we only want to allow data signed with certain digital certificates or using a certain encryption transform. You could also allow anybody to access IPSec. This is only recommended for testing. To do this, edit your policy file to contain only the following lines:

The same policy file contains two lines that start with the $ character. You need to remove these lines before using it, they are only for cvs.

A more useful policy file for this example looks like this:

Implementing this will give you a basic VPN (tunnel) using ESP only. On host A, edit /etc/isakmpd/isakmpd.conf. The sample IP address should be replaced with the external IP address of host A. Do similar for isakmpd.conf on host B. represents the external IP address for host B.

This is where you can setup the variables that will affect the main behavior of isakmpd. It is okay to use the defaults here. The Listen-on= value specifies the IP that isakmpd should listen on. Only the Internet IP of your gateway is necessary. If you have multiple external interfaces on your gateway, you could list which interfaces you want to listening on by entering them using a comma separated list.

Next, on host A, edit isakmpd.conf again.

On host B:

This section describes the IP addresses to accept in order to negotiate the phase 1 connection. Its value points to the section below (Remember that phase 1 simply authenticates the remote peer to make sure they are who they say they are). You can list multiple peers with additional lines in the format of IP_Address= <PEER-NAME>.

Next, on host A:

On host B:

This describes Phase 2 of the connection. This is the phase that determines what protocols the two peers will use to communicate.

The Connections= tag refers to the section below. It initiates the requirements or the accepted methods to set up Phase 2. This also tells ISAKMPD which connections to initiate once started. Note that you can have multiple sections as illustrated below if you are to connect with multiple peer hosts.

If you do not have the IP address of the remote host, you can specify a Default= that points to a section describing a generic entry that will be referenced by any incoming IP that is not listed in the Connections= tag.

On host A:

On host B: These represent the sections referred to by the Phase 1 section above. They each describe the requirements that the peer gateway must fulfill in order to proceed to Phase 2. There are many other options here but the ones mentioned above are the minimum requirements.

On Host A:

On Host B:

These represent the sections referred to by the Phase 2 section above. They are the individual settings that ISAKMPD must use to talk between the two gateways for the particular connection.

This is the IPSec-ID section. These entries need to exist in the isakmpd.conf files for both Host A and Host B. This example will setup for Host A (which was connected to Net-A above) and for Host B (Net-B above).

These two sections are in the conf file of each host. They are the sections referenced by the Local-ID and Remote-ID identifiers. They describe the routes that should be set up to allow traffic from one private network to another. ID-type= can be IPV4_ADDR_SUBNET or IPV4_ADDR (RFC2708 mentions more possible values. Currently only IPv4 is supported in the OpenBSD implementation. IPv6 may be supported in OpenBSD-current. )

Now, on both hosts, the sample file should read:

This section describes the requirements for the encryption methods of Phase 1 connections. The name reflects the value of Configuration= variable. As we can see here, we are stating our Domain of Interest which is IPSEC. The EXCHANGE_TYPE variable is set to ID_PROT for Phase 1, which identifies the protocols to be covered by this Authentication. Transforms= is the transform required (or assigned) for this exchange. In this case, this points to the section below in the configuration file that says we are receiving a packet encrypted with 3DES and a checksum verifiable with SHA. There are a bunch of different transforms defined inside the sample VPN-east.conf. These are provided because 3DES and SHA are not always supported across different platforms. For OpenBSD there should be no reason to change this for a basic setup. Feel free to create multiples of this section and change the transform. The only requirement is that you change the Configuration= variable.

This section describes the requirements for the encryption of the data to be sent through the VPN and is referred to by Configuration above. Note the difference between this section and the Phase 1 equivalent just above is that the EXCHANGE_TYPE is QUICK_MODE. This is always the case for Phase 2. Suites= points to a IPSec Suite section describing the different encryption schemes available between the two hosts. There is much more to be said about ISAKMP and IPSec. By using the above basic descriptions you should be able to create a simple but solid VPN that cares and feeds itself. This is the bare minimum isakmpd.conf for both hosts here.

13.8.3 - Starting isakmpd

You may wish to use


the first time you decide to run this daemon. The daemon will not be running in daemon mode but as a regular process. It will log everything to your terminal. To stop isakmpd and flush the routes, you need to kill the isakmpd process on each node, and run ipsecadm flush.

13.9 - How do I use isakmpd with X.509 certificates?

Setting up isakmpd to use certificates instead of pre-shared keys is not really that much harder in a big network with many untrusted peers than it would be with a small network. It may actually simplify configuration, and more importantly, key management.


Generating certificates.

There is a good description of how to generate keys and certificates in the README.PKI file in the isakmpd source directory. You need to have a CA key, a corresponding CA X.509 certificate, one private key for each computer on the network that will use isakmpd and one X.509 certificate for each such key.

The X.509 certificates need to have a Subject Alternative Name (SubjectAltName) extension describing the certificate holder. How to set a SubjectAltName extension using certpatch for a certificate is also described in README.PKI for the case of setting an IP address as the SubjectAltName. Using an IP address here is also the default behaviour for isakmpd.

Certpatch also support using either a FQDN (Fully Qualified Domain Name) or a UFQDN (User FQDN). An example of an FQDN might be, an example of an UFQDN would be an email address. Something like for example.

In this howto document I am going to use FQDNs as SubjectAltNames. Using IP addresses would be a bit easier since that is the default behaviour of isakmpd but it is not much of a difference as we soon will see.

To insert an FQDN SubjectAltName into a certificate one would do something like this:


Here the ca.key is the private key of the CA, thus this can only be done by whoever has access to the CA private key. The (fictional) is the FQDN to be inserted into the certificate. The originalcert.crt and newcert.crt filenames may be the same name in which case the original file will be overwritten by the new modified certificate.

Put the keys and certificates in the directories as described at the end of README.PKI. The CA key (ca.key) should be kept in some safe place if the keys are really to be seriously used.

Configuration of isakmpd

Lets now look at the /etc/isakmpd/isakmpd.conf configuration file. It was originally taken from the example file in isakmpd.conf(5) but have been heavily modified. I will also use a much shorter file here than the example file in the man-page, I have removed from that file most parts that are not needed in this setup. In part to simplify the understanding of this setup. I have also added some commenting (some comments are left as in the isakmpd.conf(5)) and changed some names. None of the domain names used here exists as far as I know.

Actually since everyone who reads this already have a working configuration for the preshared keys case (see previous section) there won't be many surprises in this file. I won't explain this in every detail, check isakmpd.conf(5) for descriptions of the parts I don't comment on.

Let's assume our setup looks something like this                           +-- 
              | |
              +-- |                                    |                                    +--

That is, two networks that should be connected using an IPSec tunnel over an otherwise insecure network. Ignore the fact that I am using IP addresses reserved for private Internets here (RFC1918), I have to use something. I won't explain how to use isakmpd in combination with NAT or similar (because I haven't tried that myself).

Now, let's look at the configuration file. This is the file for the security gateway

So far the configuration for the local system. The remote system is configured just the same way only opposite. Thus only the first part of the isakmpd.conf file differs. Let's just look at that first part of the isakmpd.conf file for the security gateway


Now that wasn't so hard, just a bit boring to read perhaps. A slightly more interesting part next.


The policy file.

Actually, the policy file might be slightly confusing for anyone who has not used it before, especially if things doesn't work as expected. The man-page isakmpd.policy(5) is not really that bad. It might perhaps be a little bit unclear in some parts but generally it's good.

The simplest possible working policy file would contain just a single line:


This basically means that that there is no policy limitations on who would be allowed to connect. Thus not a very secure setup. The authorizer tag here means the one who has the authorization to decide the policy. The special authorizer "POLICY" has the ultimate and unlimited authority on policy. Any other authorizer must first be authorized by "POLICY" to have any authority here.

There can also be a set of conditions for what is allowed. The following policy thus would mean that only someone using the ESP protocol with some real encryption would be authorized (oh well, someone using DES would also be authorized here although DES could almost be considered snakeoil today, it is left as an exercise for the reader to change this policy into not allowing DES either). Note that anyone who does encrypts with ESP would still be allowed.


It is also possible to "sublicense" authority to someone else (might be one or more entities). The simple case would be the preshared key case. In that case, anyone who knows the preshared passphrase is authorized. Thus:


This would authorize anyone who knows this passphrase to connect and comply with the conditions (but remember that the passphrase must also be set in the Authentication tag in isakmpd.conf).

Nothing difficult so far. Now to the interesting stuff. First there can be many licensees, although all must be authorized by "POLICY". Further, authorized licensees can sublicense to other licensees. A licensee can be just a string in case it is further described in the policy file:


And now to what everyone has been waiting for. Policy can also be sublicensed or delegated to a key. In this case it is usually a X.509 certificate. The simple use of certificates would be to use them like the passphrases. Just insert individual users certificates in the policy file:


Now, this is obviously a stupid idea if there are a lot of users. The certificates that isakmpd reads from the CA- and Certificate directories, and the certificates received from the peer is converted into pseudo credentials. Such certificates converted into pseudo credentials essentially would look something like:


Now note that these are not authorized by "POLICY" and thus won't have any effect without a policy authorizing them somehow. Further, this showed what happens to certificates internally. The above credential is thus not seen in the policy file. However, it is possible to sublicense to such credentials. Remember sublicensing above. It is thus possible to license all certificates that are signed by a certain CA by putting the CA certificate as a licensee to "POLICY":


Thus the above policy is a simple example of a policy that delegates to a CA. Thus any user that has a certificate that is signed by the CA that has this certificate and otherwise comply to other conditions set by the policy and the configuration file would be authorized.


Almost not secure!

Now, to be really safe, this is not enough unfortunately. There are ways to attack a security gateway that is configured in this way. If you really don't want the details, skip a to the next section now. To everyone else, let's try to understand why this is not secure. It's not hard.

Consider what information one of the isakmpds has access to at this stage. From the configuration file, isakmpd knows which IP-address its peer will send from (in the [Phase 1] section). From the information it gets at the phase 1 negotiation it knows the ID that the peer presents itself with and the certificate it gets from the peer proves that the peer really have this ID. Looks fine so far?

Well, if the ID information was the IP (the default situation if we do not provide a phase 1 ID section) everything would be fine. The CA would have tied the IP to the cert and the IP in the configuration would be all information we need. It would be possible for an imposter to use the same IP from another computer in some cases (e.g. if both computers were on the same local network and the computer that usually have this IP is down for some reason). It should however not be possible for an imposter to be able to have a certificate and a corresponding private key that (falsely) proves that this IP belongs to the imposter.

If that ever happened, the imposter would have managed to either steal the private key from the real owner of the IP, or the imposter would have managed to fool the CA into issuing a certificate containing false information somehow. If any of these things happened, then either the private key had not been protected well enough, the CA had failed to check the identity of the imposter well enough (or the ID info for the cert) or the CA private key had not been well enough protected. Since all these are prerequisites for security to work at all, none of these situations can be allowed to ever occur.

Now, in our example the situation is different. Here we actually have an FQDN in the certificate instead of an IP address. Since we still have an IP address in the [Phase 1] section this will result in a possible security problem. What now would happen during an ISAKMP phase 1 negotiation would be that we could check that the peer was sending from the expected IP (but as explained earlier that could possibly be forged in some situations). We could check that the ID our peer presents actually belongs to our peer. But what we can not check now is if that ID really is the ID we expect our peer to have, because isakmpd have never been told what ID that should be.

Someone now might say that the DNS system ties the IP to the FQDN for the host. That is true, however today's DNS system is not secure and can, under some circumstances, be fooled to give out false information (or, it could be subject to a denial of service (DoS) attack by an attacker, and the attacker's computer might be able to fake the DNS server's answer). Secure DNS will come in the future, but it is not here yet (at least most DNS servers are not secure yet), thus today, using DNS to check if the FQDN in the cert corresponds to the expected IP is no guarantee. In fact isakmpd does not check this with DNS. Even if DNS was secure, checking this would not help in the case of using an UFQDN.

Thus in the case of having an FQDN as ID, it could be possible for an attacker to get an own private key and having this key signed by the same CA that we use (but with the attacker's own FQDN, of course). Then launch a DoS attack on our peer so that it goes down (in fact, there are some flaws in the ISAKMP protocol itself that possibly could be used to launch a remote DoS against the peer an make it go down, although I don't know how sensitive isakmpd is to those attacks). Then the attacker could configure its own computer in the same way as our peer, connect it to our peers network and try to connect using its own ID, private key and certificate.

Since our own isakmpd has not been informed about what ID to our peer (and it is because the attacker is identically configured as our peer besides the certificate, ID and private key). Further our isakmpd can check that the certificate was signed by the same CA (but most CAs sign lots of certs, a cert might not be hard to get), and that the presented ID is the same as the ID in the cert. However it would not, with the configuration presented so far, check that this ID is the expected ID. Thus the attacker would be allowed to connect.


Preventing the attack.

The question now thus is, how can we inform isakmpd about what ID to expect? This is fortunately easy, and documented in the isakmpd.policy(5). We must do the check in the policy. Like this:


Now only should be able to get an IPsec connection. More allowed IDs could easily be added by adding more alternative remote_id checks, e.g. by having conditions like these in the policy:


With this policy either of, or could connect.

Some might say that is unfortunate that there has to be entire certificates inserted in the policy. It requires some work to reformat the certificates into the format in the policy and it makes the policy rather unreadable. If someone by mistake replaced a user certificate with the corresponding CA certificate somewhere in a complex policy it might cause unauthorized users to be allowed to connect in some cases and worse, it would not be easily detectable by reading through the policy file (since X.509 certificates are not in a human readable format).

Now, for the really bleeding edge people out there a solution to this problem is available. It is now possible to use the certificate Distinguished Name (DN) instead of a certificate in the policy (the corresponding certificate must of course be available from the certs or ca directories on disk so that isakmpd can find it). With this format the policy above might look like something like this instead:


Much more readable, isn't it? The information about what the exact DN for a certificate is can be found by looking at the certificate using the openssl utility. Something like:


More complicated policy configurations are of course possible but this is a start anyway, and there are another example in the next section.

This should provide the necessary information about the certificate in ca.crt. Now this is good as long as we have a small or at least reasonably small policy. It is however still not too great if we have a gigantic site with lots of users that should be allowed to connect.

Multiuser configurations and/or centrally managed authorization.

Now, lets look at some really cool features of isakmpd. Previously we assumed that the expected peer was well known and had a static IP address. This is not always the case. Lots of people use dynamically assigned IPs or use many different computers. In other cases (like for a server) we might not know for sure who wants to connect.

Therefor one very nice feature of isakmpd is the ability to use a default tag instead of an IP in the [Phase 1] section, thus allowing isakmpd negotiations from any IP. This might thus look something like this:


First, it should be said that this configuration might not be secure from DoS attacks. As said before, there are some flaws in the ISAKMP/IKE protocols. Anyway, using a default [phase 1] section also enable us to use a kind of "authorization certificates" instead.

Consider the case where we have a lot of authorized users but when we would not accept just any user. Like at a company. We would like company employees to be allowed to connect but nobody else. Now, imagine a big company where there might be thousands of employees. We might like them to all be able to connect from any computer (not only from within the company network) but everyone should not be allowed to do anything. It should now be possible to write a policy like this:


This might not be exactly how it should be. This is as far as I know completely untested (in fact, these filter conditions might not work at all as I expect). Also, a policy such as this one (in fact any with default as peer IP), would require rewrites of the isakmpd.conf file too. This would have some security implications too. Further, for this kind of connections where anyone should be allowed to connect, it would probably be desirable to log the DN of anyone who connected. Isakmpd does not yet support that to my knowledge. Also this probably could have other security implications. You are on your own, you have been warned. The basic idea should be clear anyway.

Just in case someone missed the really interesting possibilities this would have. If all computers using ISAKMP/IKE this way had a standard set of conditions for all services the users might like to use from remote, the CA could actually authorize users by just putting the right SubjectAltName extensions in their certificates. Further, the expiration time for such certificates could be set to expire relatively often although the users would be able to download new reissued certificates when their current certificate is getting old. If the users misuse their authorizations, just stop reissuing the certificates and they won't get in more after it has expired. No need to change policy files on all computers just because an employee e.g. quits their job. The same scheme should work for other purposes than ISAKMP/IPsec too (including authorization for off-line systems!) although that would require special software. In any case, any organization doing this would probably want to be their own CA.

Multiuser configurations (mobile users) like these are possible with pre-shared keys too, but then it is required that AGGRESSIVE mode is used instead of ID_PROT mode since we then must be able to choose the right password phrase based on ID since we do not know that from what the IP is in this case (in AGGRESSIVE mode the ID is sent over at an earlier stage of the negotiation, but it is sent unencrypted, thus AGGRESSIVE mode is faster because it needs fewer message exchanges, but it also is a bit less secure since the ID is sent in clear).

13.10 - What IKE clients are compatible with isakmpd?

isakmpd is the ISAKMP/Oakley key management daemon that comes with OpenBSD. We suspect that it interoperates, at least partially, with most ISAKMP implementations, but the following have actually been tested. Note that some isakmp software out there are actually based on the OpenBSD isakmp daemon.

The following MS-Windows clients have been reported to be compatible:

The following gateways/routers have been reported to be compatible:

13.11 - Troubleshooting IPSec/VPN

Your first tool for troubleshooting IPSec is tcpdump(8). Use tcpdump to look for several things.

First, if you are using tcpdump from OpenBSD, you have an enhanced version of tcpdump which can show some information about ESP and AH packets. If you are using tcpdump from OpenBSD 2.5 or on another operating system, chances are you have an older version that will simply show the protocol number for AH or ESP. (ESP is IP protocol 50, AH is 51)