0

This is a simplification of a bigger problem I'm experiencing, but I think it covers everything. The production issue is between a StrongSWAN VPN and a Juniper SSG550. The network setup is clearly not ideal, but I've inherited it and the client is unwilling to change anything unless absolutely necessary.

There's a subnet-subnet StrongSWAN VPN between an EC2 instance Bob and another non-EC2 instance George. There is also another EC2 instance Fred in the same subnet as Bob.

Bob and George are clean Ubuntu installs. 14.04 and 14.10 respectively. Fred is Oracle Linux Server 10.5

All EC2 instances have the source/destination check disabled.

----------------------                ----------------------
|                    |                |                    |
|   192.168.1.0/24   |                |    10.0.1.0/24     |
|                    |                |                    |
|  ----------------  |                |  ----------------  |
|  | Bob          |  |                |  | George       |  |
|  | 192.168.1.12 |  |   StrongSWAN   |  | 10.0.1.22    |  |
|  |              |<-------------------->|              |  |
|  | StrongSWAN   |  |                |  | StrongSWAN   |  |
|  |              |  |                |  |              |  |
|  ----------------  |                |  ----------------  |
|                    |                |                    |
|  ----------------  |                ----------------------
|  | Fred         |  |                
|  | 192.168.1.28 |  |                
|  |              |  |                
|  ----------------  |                
|                    |                
----------------------

The ipsec.conf files are as follows:

Bob:

# Connections into AWS VPC
conn %default
    ikelifetime=60m
    keylife=20m
    rekeymargin=3m
    keyingtries=1
    keyexchange=ikev1
    authby=secret

conn client
    keyexchange=ikev1
    ikelifetime=24h
    ike=aes256-sha2_256-modp1024
    esp=aes256-sha2_256-modp1024
    aggressive=no
    lifetime=1h
    leftsubnet=192.168.0.0/16
    leftid=@vpn.amazon.com
    leftfirewall=yes
    rightfirewall=yes
    right=PUBLIC_IP_OF_GEORGE
    rightsubnet=10.0.1.0/24
    rightid=@client.xxxx.com
    auto=add

George:

# Connections into AWS VPC
conn %default
    ikelifetime=60m
    keylife=20m
    rekeymargin=3m
    keyingtries=1
    keyexchange=ikev1
    authby=secret

conn client
    keyexchange=ikev1
    lifetime=24h
    ike=aes256-sha2_256-modp1024
    esp=aes256-sha2_256-modp1024
    aggressive=no
    leftfirewall=yes
    leftsubnet=10.0.1.0/24
    leftid=@client.xxxx.com
    right=PUBLIC_IP_OF_BOB
    rightsubnet=192.168.0.0/16
    rightid=@vpn.amazon.com
    auto=add

I also added a route for Fred to George.

Fred> route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.0.1.0        192.168.1.12    255.255.255.0   UG    0      0        0 eth0
...

The problem I have is that traffic from George to Fred doesn't get there. Traffic from Fred to George does. Bob and Fred can both ping George. George can ping Bob.

I set up a listening socket on port 4000 on all three instances. Bob and Fred can both connect to the one on George and George can connect to the one on Bob. Obviously Bob and Fred can connect to each other.

Here's a tcpdump for the connection from George to Fred as seen on Bob.

Bob> tcpdump port 4000
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
11:55:37.791967 IP 10.0.1.22.46014 > 192.168.1.28.4000: Flags [S], seq 2922469699, win 29200, options [mss 1460,sackOK,TS val 3459391 ecr 0,nop,wscale 7], length 0
11:55:37.792052 IP 10.0.1.22.46014 > 192.168.1.28.4000: Flags [S], seq 2922469699, win 29200, options [mss 1460,sackOK,TS val 3459391 ecr 0,nop,wscale 7], length 0
11:55:38.792116 IP 10.0.1.22.46014 > 192.168.1.28.4000: Flags [S], seq 2922469699, win 29200, options [mss 1460,sackOK,TS val 3459641 ecr 0,nop,wscale 7], length 0
11:55:38.792188 IP 10.0.1.22.46014 > 192.168.1.28.4000: Flags [S], seq 2922469699, win 29200, options [mss 1460,sackOK,TS val 3459641 ecr 0,nop,wscale 7], length 0
11:56:07.434645 IP 10.0.1.22.46015 > 192.168.1.28.4000: Flags [S], seq 459738226, win 29200, options [mss 1460,sackOK,TS val 3466802 ecr 0,nop,wscale 7], length 0
11:56:07.434722 IP 10.0.1.22.46015 > 192.168.1.28.4000: Flags [S], seq 459738226, win 29200, options [mss 1460,sackOK,TS val 3466802 ecr 0,nop,wscale 7], length 0
11:56:08.436531 IP 10.0.1.22.46015 > 192.168.1.28.4000: Flags [S], seq 459738226, win 29200, options [mss 1460,sackOK,TS val 3467052 ecr 0,nop,wscale 7], length 0
11:56:08.436601 IP 10.0.1.22.46015 > 192.168.1.28.4000: Flags [S], seq 459738226, win 29200, options [mss 1460,sackOK,TS val 3467052 ecr 0,nop,wscale 7], length 0
^C
8 packets captured
8 packets received by filter
0 packets dropped by kernel

Nothing is seen on Fred.

The iptables rules for the two EC2 instances are as follows:

Bob:

Bob> iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  10.0.1.0/24          192.168.0.0/16       policy match dir in pol ipsec reqid 1 proto esp
ACCEPT     all  --  192.168.0.0/16       10.0.1.0/24          policy match dir out pol ipsec reqid 1 proto esp

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Fred:

Fred> iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Any suggestions on what the problem might be would be much appreciated. At the moment I'm not sure if it's an EC2 problem or a networking issue.

sunaxi
  • 1
  • 3
  • Doesn't "Bob and Fred can both ping George" imply that Fred is actually able to receive packets from George (i.e. ICMP replies)? Or are you only referring to the ICMP requests? Are there any NAT rules on either host (`iptables -t nat -L`)? – ecdsa May 27 '15 at 15:35
  • There are no NAT rules on either side, but you've hit the nail on the head. This is what is confusing me. Bob and Fred can both ping/telnet to George, but George can only telnet/ping to Bob. – sunaxi May 27 '15 at 16:20
  • Enabling masquerading appears to be helping. I'll test some more. `Bob> iptables -t nat -A POSTROUTING -s 10.0.1.0/24 -o eth0 -m policy --dir out --pol ipsec -j ACCEPT` `Bob> iptables -t nat -D POSTROUTING -s 10.0.1.0/24 -o eth0 -j MASQUERADE` – sunaxi May 27 '15 at 17:44
  • If `Bob` and `George` are the servers, where are the StrongSWAN VPN and Juniper SSG550 in this? Can you print the routes for all 3 systems? – CIA May 27 '15 at 18:01
  • StrongSWAN is running on Bob and George. In production George is behind the Juniper and not running StronSWAN. To try and reduce the problem I've replicated the problem using StrongSWAN. This has taken the Juniper out of the list of possible problems. There's no special routing going on now (apart from whatever StrongSWAN adds) except that a route for 10.0.1.0/24 has been added to the VPC that Bob and Fred are in. The route sends all 10.0.1.0/24 traffic to Bob. – sunaxi May 27 '15 at 18:24
  • Do you see an ARP request sent from Bob for Fred's IP when George tries to ping Fred? And a corresponding response? Or experimentally, is George able to ping Fred right after there was a successful ping from Fred to George (or while such pings are still going on)? – ecdsa May 28 '15 at 09:24

1 Answers1

0

I managed to solve my problem with a little help from the comments above, tcpdump and Split Tunneling.

There were actually two issues, although only one in this test scenario.

I needed some iptables masquerading magic

Bob> iptables -t nat -A POSTROUTING -s 10.0.1.0/24 -m policy --dir out --pol ipsec -j ACCEPT
Bob> iptables -t nat -D POSTROUTING -s 10.0.1.0/24 -j MASQUERADE

In production there was an added complication that the EC2 side of the VPN (whilst in a 192.168.1.0/24 subnet) was the 172.22.70.192/28 subnet. This needed some routing magic to ensure that packets from Fred (eth0 10.0.1.28, eth0:0 172.22.70.194) to George arrived at Bob (eth0 10.0.1.12, eth0:0 172.22.70.193) with the correct IP address.

Fred> ip route add 10.0.1.0/24 via 192.168.1.12 dev eth0:0 src 172.22.70.194

Now everything works.

sunaxi
  • 1
  • 3