0

First, I am aware of creating a VXLAN interface with tag based on ip command:

ip link add vxlan-br0 type vxlan id <tag-id> group <multicast-ip> local <host-ip> dstport 0

But it is useless for my actual demand, and my demand is to isolate multiple docker containers using different tags, something like:

brctl addif br1 veth111111 tag=10 # veth111111 is the netdev used by docker container 1 brctl addif br1 veth222222 tag=20 # veth222222 is the netdev used by docker container 2 brctl addif br1 veth333333 tag=10 # veth111111 is the netdev used by docker container 3

I want to isolate container 2 from container 1 and 3, and don't isolate communication bewteen container 1 and 3. How to achieve this?

ghostplant
  • 87
  • 1
  • 7

1 Answers1

0

Adding two bridge networks will provide isolation. 

docker create network net1
docker create network net2

Then start some containers

docker run -d --name one --net net1 busybox sleep 600
docker run -d --name two --net net2 busybox sleep 600
docker run -d --name three --net net1 busybox sleep 600

one and three will communicate as they are attached to the same bridge

docker exec one ping three
docker exec three ping one

Others will fail as they cross networks/bridges

docker exec one ping two
docker exec two ping one
docker exec three ping two

You'll notice docker provides host/name resolution inside a network so it's actually the host name resolution that is failing above. IP's are not routed between bridges either. 

$ docker exec three ip ad sh dev eth0
17: eth0@if18: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue 
    link/ether 02:42:ac:14:00:03 brd ff:ff:ff:ff:ff:ff
    inet 172.20.0.3/16 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe14:3/64 scope link 
       valid_lft forever preferred_lft forever

Ping two

$ docker exec three ping -c 1 -w 1 172.21.0.2
PING 172.21.0.2 (172.21.0.2): 56 data bytes

--- 172.21.0.2 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

Ping one

docker exec three ping -c 1 -w 1 172.20.0.2
PING 172.20.0.2 (172.20.0.2): 56 data bytes
64 bytes from 172.20.0.2: seq=0 ttl=64 time=0.044 ms

This setup will work with the overlay networking driver as well but that is more complex to setup.

Matt
  • 68,711
  • 7
  • 155
  • 158
  • A good ideal. However, this approach requires to create O(|Group|) bridges. In the worst case, |Group| = |Containers|. Do you know the maximum number of bridges can be created on one host? Is there any solution to create O(1) bridge for common use and use VXLAN tag to isolate different groups? – ghostplant Jun 11 '16 at 09:50
  • @ghostplant For single container isolation, use `--icc=false` on the default network. One bridge has a limit of 1023 interfaces. Apparently things start [slowing down in Linux past 9600 interfaces](https://support.cumulusnetworks.com/hc/en-us/articles/216420547-Calculating-the-Limitation-of-a-Linux-Bridge-In-Traditional-Mode) but I've never pushed networking that far. You should get more bridges+interfaces than you will interfaces on a bridge. I would also caution against custom networking so you can use Docker tools like compose, swarm, Kuberenetes, ECS etc easily. – Matt Jun 11 '16 at 11:44