In Windows, you enable IPSEC via security policy. Either you can edit the local security policy of individual machines or you can use domain group policy (WhatIsGroupPolicy) to configure IPSEC in a consistent way on a whole group of machines, which is a great way to go. Either way you do it, the procedure for configuring IPSEC is the same.

If you look at the security policy for a machine, you'll see a section called IP Security Policies. Figure 69.1 shows what mine looks like on my Windows XP box. It lists several optional policies (strategies, if you will) for applying IPSEC. Notice that they are all disabled by default.

Figure 69.1 Examining IPSEC policy

The simplest way to turn on IPSEC support between two computers is to enable the Server policy on the server machine and the Client policy on the client machine. The former says that the server will negotiate IPSEC with the intention of establishing a secure channel, knowing, however, that not all clients will support it. Thus the Server policy allows the client to decide whether a secure connection will be used. The Client policy simply says that, if a server supports IPSEC, the client will agree to use it. You can enable any one of these policies by right-clicking it and selecting Assign, but only one policy may be enabled at any given time, and you can similarly Unassign the active policy to turn off IPSEC support.

A much more secure option is to use the Secure Server policy on the server, which will force all IP communication to be secured with IPSEC or be rejected. Only client machines with an IPSEC policy enabled will be able to communicate with the server.

But saying, "All IP communication will be secured with IPSEC" is pretty meaningless. Remember that IPSEC is a collection of several protocols (WhatIsIPSEC). To understand what will happen when a particular client connects to a server, you need to drill down into these policies a bit. For example, if you double-click the Secure Server policy, you'll find three different strategies for dealing with peers depending on the type of traffic. Double-click the All IP Traffic option and then click on the Filter Action tab of the resulting dialog. From there, double-click the Require Security filter action (it should be the one whose radio button is checked). You'll see a dialog like the one I've shown in Figure 69.2.

Figure 69.2 Options for negotiating security

Basically IPSEC allows three options for any given connection to a peer: permit raw (unsecured) communication, block communication completely, or negotiate a secure channel with the peer. Note that the default list of negotiated protocols and algorithms is quite long. This is one place where you can afford to lock things down a bit more. If you're simply locking down machines within your own organization, and you know they'll all be using Microsoft's implementation of IPSEC, by all means restrict the flavors of IPSEC that you're willing to negotiate. Notice that the list contains two encryption and hash algorithms, and if you know much at all about cryptography, you'll know that 3DES is a known secure algorithm in broad use today whereas its older cousin, DES, is easily broken. You'll also know that the SHA1 algorithm is preferable to MD5. So remove any options here that include the two weaker algorithms. That will leave you with one option: the ESP protocol using 3DES encryption with HMAC-SHA1 integrity protection. You can do the same thing for the Client policy on client machines, but there you'll also see the AH protocol being negotiated as a last resort. Remove any entries that use AH because it doesn't support encryption, as I discussed in (WhatIsIPSEC).

Another option you should enable is called "Session key perfect forward secrecy (PFS)." This means that, when session keys are swapped out (this happens after a set number of bytes have been exchanged or a fixed amount of time goes by), the new session key won't simply be shipped across the existing secure connection. This would enable an attacker who has recorded all communications between two machines to break one session key and therefore immediately unwind all the rest of the session keys exchanged over time. With PFS, the new session key is exchanged using a protocol called Diffie-Hellman key exchange (Ferguson and Schneier 2003), in which an eavesdropper must break each session key individually. This is a good thing, so be sure to enable it.

In many systems, simply turning on the Secure Server and Client policies and tweaking them as I've suggested will suffice. It's also possible to have the server accept unsecured traffic from certain subnets by adding custom IP filters, but this is more dangerous than simply locking down all traffic to the server.

Try it! Configure a couple of computers in the lab domain environment to use IPSEC and use a network sniffer such as netmon.exe on the server to sniff the packets. If you do something simple, such as make an HTTP request, without IPSEC you'll see the request going across in the clear. With IPSEC, however, the request and response will be encrypted.

Another good resource with links to information about IPSEC in Windows can be found in (Meier et al.).

PortedBy Steve Johnson