0

I have an Ubuntu host system that is running a MAC-VLAN virtual interface on top of eth0 interface, ipv4 routing is enabled. Also, this system has a Docker (LXC) container running:

docker0   Link encap:Ethernet  HWaddr d6:00:77:0f:ab:9e  
      inet addr:172.17.42.1  Bcast:0.0.0.0  Mask:255.255.0.0
      inet6 addr: fe80::d8be:a9ff:fe59:eba6/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:142270 errors:0 dropped:0 overruns:0 frame:0
      TX packets:288893 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0 
      RX bytes:6599880 (6.5 MB)  TX bytes:429869675 (429.8 MB)

eth0      Link encap:Ethernet  HWaddr 00:1e:c9:4d:15:bc  
      inet addr:10.0.1.206  Bcast:10.0.1.255  Mask:255.255.255.0
      inet6 addr: fe80::21e:c9ff:fe4d:15bc/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:498286 errors:0 dropped:0 overruns:0 frame:0
      TX packets:178679 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:571255676 (571.2 MB)  TX bytes:16081465 (16.0 MB)

macvlan0  Link encap:Ethernet  HWaddr 1a:11:5e:36:a0:16  
      inet addr:10.0.1.86  Bcast:10.0.1.255  Mask:255.255.255.0
      inet6 addr: fe80::1811:5eff:fe36:a016/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:85887 errors:0 dropped:0 overruns:0 frame:0
      TX packets:103 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0 
      RX bytes:5623532 (5.6 MB)  TX bytes:33642 (33.6 KB)

Docker container has an IP from the 172.17.0.0/16 network:

eth0      Link encap:Ethernet  HWaddr 36:6a:ef:b5:a8:e8  
      inet addr:172.17.0.37  Bcast:172.17.255.255  Mask:255.255.0.0
      inet6 addr: fe80::346a:efff:feb5:a8e8/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:5928 errors:0 dropped:0 overruns:0 frame:0
      TX packets:2675 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:8899041 (8.8 MB)  TX bytes:159716 (159.7 KB)

What I want to do is to make the container to be visible to the outside machines as 10.0.1.86, i.e. the IP assigned to macvlan0, so all the packets that go to the IP 10.0.1.86 automatically go to 172.17.0.37.

My guess is that I should setup NAT somehow, and here's what I tried to do, judging by the articles I found in the internet:

iptables -t nat -A PREROUTING -i macvlan0 -j DNAT --to-destination 172.17.0.37
iptables -t nat -A POSTROUTING -o macvlan0 -j MASQUERADE

However, when I'm trying to ssh to 10.0.1.86, I'm still logging in to the host system.

What am I doing wrong here?

UPD: maybe, I should just set the IP addresses for container's eth0 and host system's docker0 to something from 10.0.1.0/24 instead, and query the container directly by its IP address?

Andrew Domaszek
  • 5,163
  • 1
  • 15
  • 27
demeshchuk
  • 111
  • 2
  • Your network config seems overly complicated; why do you need a macvlan interface? Are you trying to prevent crosstalk from one VM to another or from the VM to the host system? If you don't know why you are using the macvlan, have you considered making a bridge device? How does your LXC traffic find its way to the macvlan0 interface as opposed to eth0? Are you using policy routing or bridging for this? – Andrew Domaszek Sep 09 '13 at 06:21
  • ... or was all of this created by docker? If it was created by docker, you should use it to manage your incoming ports ([docker incoming port redirection](http://docs.docker.io/en/latest/use/port_redirection/)); all outgoing connections should already be properly NATted. To verify, post the results of `iptables-save`, `ip route show table all`, `ip rule show`. – Andrew Domaszek Sep 09 '13 at 06:41
  • The bridge docker0 is created by Docker itself, unfortunately. But yes, I feel that this is too complicated too. Feels like if docker0 had a dhcp address in the same subnet as the host system, a lot would be easier. Well, that's what I'll try to do by hacking Docker itself. – demeshchuk Sep 09 '13 at 19:49
  • My advice is remove macvlan0. Use docker's -p option to forward relevant ports as needed. If docker creates docker0, you may want to consider adding eth0 to docker0 and giving docker0 an IP on your main network. Hopefully it doesn't get confused as to what that means and try to assign in-use IP addresses. Poke around in the docker docs (yes, I know they are hard to navigate) to see if they support off-machine bridges with no NAT. – Andrew Domaszek Sep 09 '13 at 20:04

1 Answers1

0

So, I almost figured that out.

First of all, macvlan0 interface is really not needed.

Secondly, docker0 should be configured using DHCP instead.

Lastly, lxc.network.ipv4 should be empty for the container, and lxc.network.hwaddr should be set to some unique value.

Unfortunately, this works only with vanilla lxc containers, docker does some extra magic upon every container run.

demeshchuk
  • 111
  • 2