3

I asked about this on the Smoothwall Express forums but they are understandably reluctant to provide to much help. After all, they really want to sell the commercial version.

I need to provide an L2TP/IPSEC VPN for remote support of some new machinery we're getting soon (no choice about that part). As the remote user also needs to be authenticated against Active Directory I need to run the VPN on our Windows (2003) server, rather than directly on the firewall.

Because Smoothwall Express has no provision for forwarding protocols from it's normal management interface this needs to be done by editing the iptables settings in rc.firewall.up. Herein lies my problem. I don't know iptables and feel that even if I take the time to read up about it the risks of me screwing it up are significant.

The following section currently exists in rc.firewall.up:

# IPSEC
/sbin/iptables -N secin
/sbin/iptables -A secin -i ipsec0 -j ACCEPT
/sbin/iptables -A INPUT -j secin

/sbin/iptables -N secout
/sbin/iptables -A secout -i ipsec0 -j ACCEPT
/sbin/iptables -A FORWARD -j secout

[some unrelated stuff here]

# IPSEC
/sbin/iptables -N ipsec
/sbin/iptables -A ipsec -p udp --destination-port 500 -j ACCEPT
/sbin/iptables -A ipsec -p udp --destination-port 4500 -j ACCEPT
/sbin/iptables -A ipsec -p 50 -j ACCEPT
/sbin/iptables -A ipsec -p 51 -j ACCEPT
/sbin/iptables -A block -i ppp0 -j ipsec
/sbin/iptables -A block -i ippp0 -j ipsec
if [ "$RED_DEV" != "" ]; then
    /sbin/iptables -A block -i $RED_DEV -j ipsec
fi

Unless I'm mistaken that second bock will cause the traffic I want forwarded to be swallowed instead. I tried commenting out that second block and adding this one, which is adapted from the lines required for PPTP passthrough (which does work) and what I've been able to obtain from various sources:

# L2TP/IPSEC
/sbin/iptables -N l2tp
/sbin/iptables -A l2tp -p udp --destination-port 500 --dst 192.169.0.7 -j ACCEPT
/sbin/iptables -A l2tp -p udp --destination-port 1701 --dst 192.169.0.7 -j ACCEPT
/sbin/iptables -A l2tp -p udp --destination-port 4500 --dst 192.169.0.7 -j ACCEPT
/sbin/iptables -A l2tp -p 50 --dst 192.169.0.7 -j ACCEPT
/sbin/iptables -A l2tp -p 51 --dst 192.169.0.7 -j ACCEPT
/sbin/iptables -I FORWARD -j l2tp
/sbin/iptables -t nat -N l2tp
/sbin/iptables -t nat -A l2tp -i $RED_DEV -p tcp --dport 500 -j DNAT --to 192.169.0.7:500
/sbin/iptables -t nat -A l2tp -i $RED_DEV -p tcp --dport 1701 -j DNAT --to 192.169.0.7:1701
/sbin/iptables -t nat -A l2tp -i $RED_DEV -p tcp --dport 4500 -j DNAT --to 192.169.0.7:4500
/sbin/iptables -t nat -A l2tp -i $RED_DEV -p 50 -j DNAT --to 192.169.0.7
/sbin/iptables -t nat -A l2tp -i $RED_DEV -p 51 -j DNAT --to 192.169.0.7
/sbin/iptables -t nat -A PREROUTING -j l2tp

.. but that doesn't work. Where did I stuff up?

Incidentally, $RED_DEV translates to the public facing interface and obviously 192.168.0.7 is my VPN server.

Update:

It turns out that the above settings do work - sort of. I'm testing this from home, where I have a MacBook and a several of Windows XP machines. I can connect just fine from the Mac but cannot connect using any of the Windows machines. Unfortunately, the machinery company that requires this VPN for remote support uses only Windows XP. :(

John Gardeniers
  • 27,458
  • 12
  • 55
  • 109
  • For what it is worth I found that NAT for VPN servers was pretty hopeless. The reason is just about every client will be using NAT as well -- so with the double NAT scenerio I had a set up that worked with a ratio of about 1 out of 3 people. – Kyle Brandt Mar 07 '11 at 21:12
  • @Kyle, unfortunately I don't see any other viable options. The company we are buying the machinery from will *only* use an MS L2TP/IPSEC client with a pre-shared key. As we need their services far more than they need our business we get no say in this and they've made it quite clear that it's their way or no way (I'm not involved in such decisions). This VPN will not be used for any other purposes, so if you have any suggestions that could possibly make this work I'm eager to see them. – John Gardeniers Mar 07 '11 at 21:19
  • You can put a public IP on the VPN server itself and just forward the traffic instead of NATing it. NAT may be fine too -- just saying it gave me a lot of trouble in the case. – Kyle Brandt Mar 07 '11 at 21:22
  • @Kyle, you're not the first to tell me this might now work. :( – John Gardeniers Mar 07 '11 at 21:25

2 Answers2

4

As noted in the update to the question, even with double NAT the VPN is working from a Mac to Windows but not Windows to Windows. It seems it's really only a matter of coming up with the correct search term to find a solution.

As per this KB article, on the Windows XP client machine we need to create a new DWORD value named AssumeUDPEncapsulationContextOnSendRule in HKLM\System\CurrentControlSet\Services\IPSec. For double NAT it needs a value of 2.

Now I need to find a solution for Windows 7, because it's inevitable that sooner or later I'm going to have to deal with that.

Update

For the benefit of anyone with the same issue, the fix for Windows 7 is to create a new DWORD value named AssumeUDPEncapsulationContextOnSendRule in HKLM\SYSTEM\CurrentControlSet\services\PolicyAgent. Again, for double NAT it needs a value of 2.

John Gardeniers
  • 27,458
  • 12
  • 55
  • 109
-2

Here is a guide to make sure the client ports are open on the windows machine itself.

https://www.magnumvpn.com/setup-windows-10-firewall-l2tp.html

privacychair
  • 17
  • 2