Docker Host with multiple VLANs

Question

Background Information

I have a server with one physical network interface that is running Docker. This interface is configured as a 802.1Q trunk. To avoid asymetric routing I configured routing tables for each subnet. Thats my interfaces /etc/network/interfaces :

auto enp3s0
iface enp3s0 inet dhcp
    post-up ip route add 192.168.1.0/24 dev enp3s0 table 1
    post-up ip route add default via 192.168.1.1 table 1
    post-up ip rule add from 192.168.1.0/24 table 1 priority 101
    post-up ip route flush cache
    pre-down ip rule del from 192.168.1.0/24 table 1 priority 101
    pre-down ip route flush table 1
    pre-down ip route flush cache

auto enp3s0.2
iface enp3s0.2 inet dhcp
        hwaddress ether 00:11:22:33:44:55
        post-up ip route add 192.168.2.0/24 dev enp3s0.2 table 2
        post-up ip route add default via 192.168.2.1 table 2
        post-up ip rule add from 192.168.2.0/24 table 2 priority 102
        post-up ip route flush cache
        pre-down ip rule del from 192.168.2.0/24 table 2 priority 102
        pre-down ip route flush table 2
        pre-down ip route flush cache

auto enp3s0.4
iface enp3s0.4 inet dhcp
        hwaddress ether 00:11:22:33:44:56
        post-up ip route add 192.168.4.0/24 dev enp3s0.4 table 4
        post-up ip route add default via 192.168.4.1 table 4
        post-up ip rule add from 192.168.4.0/24 table 4 priority 104
        post-up ip route flush cache
        pre-down ip rule del from 192.168.4.0/24 table 4 priority 104
        pre-down ip route flush table 4
        pre-down ip route flush cache
...

This setup works fine, if I start container with the --net=host parameter. The containers are accessible from each subnet/vlan.

The Problem

I would like to have more control about the ports and the accessibility (not every docker should be reachable in every subnet). If I use the parameter -p (e.g. -p 3777:3777) the dockers are not reachable anymore.

This guide https://hicu.be/docker-networking-macvlan-vlan-configuration adress a simliar problem, but I do not want to extend my vlans to docker and assign an IP on each docker instance. This is to much.

Desired solution

My server has an IP in every subnet/vlan, 192.168.1.199 (native vlan / mgmt) 192.168.2.199 (vlan2) 192.168.4.199 (vlan4)

I would like to start dockers with the -p paramenter and will choose on wich interface it is accessible. e.g. docker run --p 9000:9000 --name portainer ... and it should only accessible through 192.168.1.199:9000

Maybe my ip route / ip rule settings are not well configured or/and I need a docker bridge for each subnet...but that's the point where I can not get any further. Up to now, if I choose the --p parameter and the docker is connected to the default docker bridge...the docker is not accessible at all.

Do you have any idea?

Greets, Mark

Edit: The container portainer_test is not accessible via 192.168.4.199:9001

   mark@server:~/docker$ docker ps -a
        CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS                  PORTS                          NAMES
        359ebfd027b2        portainer/portainer         "/portainer -H unix:…"   21 minutes ago      Up About a minute       192.168.4.199:9001->9000/tcp   portainer_test
        9d523a8b22e4        eclipse-mosquitto           "/docker-entrypoint.…"   10 days ago         Up 16 hours                                            mosquito
        a2eeb9582838        portainer/portainer         "/portainer"             10 days ago         Up 16 hours                                            portainer
        f4ef7570cea2        symcon/symcon:stable        "/usr/bin/symcon"        10 days ago         Up 16 hours                                            symcon
        ae43e8be871f        jacobalberty/unifi:stable   "/usr/local/bin/dock…"   10 days ago         Up 16 hours (healthy)                                  unifi
        mark@server:~/docker$ sudo netstat -tulpn | grep LISTEN
        tcp        0      0 127.0.0.1:27117         0.0.0.0:*               LISTEN      23374/bin/mongod
        tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      30474/systemd-resol
        tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1592/sshd
        tcp        0      0 0.0.0.0:1883            0.0.0.0:*               LISTEN      22212/mosquitto
        tcp        0      0 0.0.0.0:3777            0.0.0.0:*               LISTEN      22247/symcon
        tcp        0      0 192.168.4.199:9001      0.0.0.0:*               LISTEN      18622/docker-proxy
        tcp6       0      0 :::8843                 :::*                    LISTEN      22511/java
        tcp6       0      0 :::8880                 :::*                    LISTEN      22511/java
        tcp6       0      0 :::8080                 :::*                    LISTEN      22511/java
        tcp6       0      0 :::8443                 :::*                    LISTEN      22511/java
        tcp6       0      0 :::1883                 :::*                    LISTEN      22212/mosquitto
        tcp6       0      0 :::6789                 :::*                    LISTEN      22511/java
        tcp6       0      0 :::9000                 :::*                    LISTEN      22273/portainer

score 0 · Answer 1 · answered Apr 16 '20 at 10:56

It may be too late, but I post what has worked on my side with VLAN trunk coming into my NAS running docker....

You need to create your own BRIDGE network => https://docs.docker.com/network/network-tutorial-standalone/#use-user-defined-bridge-networks The default bridge created binds to all IPs of the host (0.0.0.0), which is your issue.

Make sure you use this option during its creation: com.docker.network.bridge.host_binding_ipv4 with your desired server IP address (in your case 192.168.1.199, so that the bridge created only binds to this IP)

Then just specify (in your example) that Portainer container uses this user-defined bridge network (--network when you create/run the container).

Else, you may also look into macvlan network type. Good luck :-)

score 0 · Answer 2 · answered Jan 24 '19 at 12:09

0

for the -p parameter, you can use ip:hostPort:containerPort format, for example

docker run -p 192.168.1.199:9000:9000 ...

More info about available formats are in the doc

answered Jan 24 '19 at 12:09

Quantim

1,358
10
15

Hi, I tried this one, but it dont work: "ERR_CONNECTION_TIMED_OUT" CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 359ebfd027b2 portainer/portainer "/portainer -H unix:…" 8 seconds ago Up 7 seconds 192.168.4.199:9001->9000/tcp portainer_test – Mark Jan 24 '19 at 13:21

Docker Host with multiple VLANs

2 Answers2