Allow Docker containers to use IPSEC VPN on host

Question

I have Docker and an IPSEC VPN tunnel on my workstation, but the containers can't access hosts behind the VPN. Is there anything I can do, e.g. with IPTables or routes to allow access? [Note: I'm currently running these tests from within a libvirt-hosted VM, then libvirt bridge has address 192.168.122.1]

My networking setup looks like this:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:0f:f2:bb brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.87/24 brd 192.168.122.255 scope global dynamic ens3
       valid_lft 3490sec preferred_lft 3490sec
    inet6 fe80::5054:ff:fe0f:f2bb/64 scope link 
       valid_lft forever preferred_lft forever
3: tap0: <BROADCAST,UP,LOWER_UP> mtu 1380 qdisc pfifo_fast state UNKNOWN qlen 500
    link/ether d6:2b:f6:24:c5:1c brd ff:ff:ff:ff:ff:ff
    inet 172.20.1.29/24 brd 172.20.1.255 scope global tap0
       valid_lft forever preferred_lft forever
    inet6 fe80::d42b:f6ff:fe24:c51c/64 scope link 
       valid_lft forever preferred_lft forever
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether 02:42:24:7f:6a:1a brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:24ff:fe7f:6a1a/64 scope link 
       valid_lft forever preferred_lft forever

My routes look like this (VPN endpoint IP obfuscated):

default via 192.168.122.1 dev ens3  proto static  metric 100 
10.0.0.0/8 via 172.20.1.29 dev tap0  proto static 
10.11.12.13 via 192.168.122.1 dev ens3  proto static 
172.17.0.0/16 dev docker0  proto kernel  scope link  src 172.17.0.1 
172.20.1.0/24 dev tap0  proto kernel  scope link  src 172.20.1.29 
192.168.122.0/24 dev ens3  proto kernel  scope link  src 192.168.122.87  metric 100

I can see some existing masq rules, the packet counter on the first rule increments when I send test pings out from a container:

[robin@rhel72 ~]$ sudo iptables -t nat -L -n -v --line-numbers | sed -n '/^Chain POSTROUTING /,/^$/ p'
Chain POSTROUTING (policy ACCEPT 4 packets, 534 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1       63  4290 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0           
2      473 30282 POSTROUTING_direct  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
3      473 30282 POSTROUTING_ZONES_SOURCE  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
4      473 30282 POSTROUTING_ZONES  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Finally, I can see what looks like a partial response when I tcpdump the connection whilst running ping -c1 from inside the container:

[robin@rhel72 ~]$ sudo tcpdump -i any -n 'icmp'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
09:55:01.919214 IP 172.17.0.2 > 10.60.1.201: ICMP echo request, id 27, seq 1, length 64
09:55:01.919214 IP 172.17.0.2 > 10.60.1.201: ICMP echo request, id 27, seq 1, length 64
09:55:01.940613 IP 10.60.1.201 > 172.20.1.29: ICMP echo reply, id 27, seq 1, length 64

EDIT 1

Routing table inside the Docker container is:

[root@451c1c9c708c /]# ip route
default via 172.17.0.1 dev eth0 
172.17.0.0/16 dev eth0  proto kernel  scope link  src 172.17.0.2

It may be worth noting that I'm running the Docker daemon using the --icc=false flag to isolate containers.

you need to masquerade the docker interface to the interface allowed across the VPN, there seems to be a couple of options:. https://docs.docker.com/v1.5/articles/networking/ — Sum1sAdmin, Apr 18 '16 at 11:25

Christopher Brichford · Answer 1 · 2016-04-19T22:49:37.320

7

I wrote a python program to wrap ipsec that installs iptables entries to allow docker containers to talk to the VPN tunnel:

https://github.com/cbrichford/docker-ipsec

Instead of doing: ipsec up

You do: docker-ipsec up

This script might need some work for use with the latest docker networking features, but worked with the old default docker0 bridge.

If you don't want to use my script to edit iptables here is how you construct the iptables command:

Figure out the virtual IP address of the host in the VPN. Call this virtualIP.
Figure out the CIDR block for IP address in the VPN. If IP address in the VPN are always of the form 10.10.X.X then your CIDR block would be 10.10.0.0/16. Call this vpnSubnet
Figure out the default route interface. This will be something like "eth0". This is the command I use to find it: sudo ip route show | grep -e "^default" | awk -- "{ print \$5 }". Call this defaultRouteInterface
Figure out the CIDR block for the docker network you want to grant access to your VPN. I this command to find it: sudo ip route show | grep -e "[[:space:]]dev[[:space:]]docker" | awk -- "{ print \$1 }". Call this dockerSubnet.

The iptables command you need to run is: sudo \ iptables \ -j SNAT \ -t nat \ -I POSTROUTING 1 \ -o ${defaultRouteInterface} \ -d "${vpnSubnet}" \ -s "${dockerSubnet}" \ --to-source "${virtualIP}"

edited Apr 19 '16 at 22:49

answered Apr 18 '16 at 17:37

Christopher Brichford

71
2

Hi Christopher. This project looks great, my only issue is that I'm not using ipsec! Can you paste some raw `iptables` commands in please? – Robin Apr 19 '16 at 10:31
p.s. Looks like you've got a typo here : https://github.com/cbrichford/docker-ipsec/blob/master/docker_ipsec/docker-ipsec.py#L58 :) – Robin Apr 19 '16 at 10:45
@Robin, I edited my answer with a procedure to figure out the right iptables command to run. – Christopher Brichford Apr 19 '16 at 22:51
@Robin thanks for pointing out that typo. It is now fixed. – Christopher Brichford Apr 19 '16 at 22:54
thanks for expanding your answer, sadly this hasn't solved my issue. My iptables command is `iptables -t nat -I POSTROUTING 1 -o ens3 -d 10.0.0.0/8 -s 172.17.0.0/16 -j SNAT --to-source 172.20.1.29`. I'm continuing to see the same bahiour listed in my question, i.e. the ping response packet arriving back at the 172.20.1.29 but not being delivered to the container. I'm going to try and get IPSec running (Libreswan) so I can use your script as-is. – Robin Apr 21 '16 at 13:10
Managed to boil that down to the formula in my answer below. Adding a new answer so that the formula can be better formatted. – CodeMedic Oct 22 '16 at 02:21

CodeMedic · Answer 2 · 2016-10-22T14:08:18.183

Based on @ChristopherBrichford's answer, I managed to boil the formula down to the command below. This is based on my limited understanding of various moving parts involved.

sudo iptables -j SNAT -t nat -I POSTROUTING 1 \
    -o $(ip route show | grep -e "^default" | awk -- "{ print \$5 }") \
    -d $(ip route list table 220 | grep -o '^[0-9.]*/[0-9]*') \
    -s $(ip route show | grep -e ":space:dev:space:docker" | awk -- "{ print \$1 }") \
    --to-source $(ifconfig | grep -o 'P-t-P:[^ ]*' | cut -d: -f2)

A more usable and updated version is available as a gist.

Allow Docker containers to use IPSEC VPN on host

2 Answers2