4

I would like to configure Docker containers to look and feel like a real system by assigning them static IP addresses on my network.

Currently I have a network bridge setup (br0) on the host. I have DOCKER_OPTS="-b=br0" configured in the /etc/default/docker file. I'm able to assign static addresses to each of the containers via their /etc/network/interface files.

The problem is Docker will continue to assign IP addresses on its own. (How to I stop this?) Even more troubling is that it will now use the IP scheme of the bridged network. Which will cause networking issues as it can/will assign addresses that are already in use on the network.

Issues:
Network hanging/pausing
The responding address while ping'ing container, can change.

Error messages such as this are seen in the logs of the host:
kernel: [31912.876161] br0: port 3(vethb228701) entered disabled state
kernel: [31913.441517] device veth122d9f8 left promiscuous mode
kernel: [22491.609856] audit: type=1400 audit(1434148604.621:124): apparmor="DENIED" operation="getattr" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/ntpd" name="var/lib/docker/aufs/diff/eb00895db3d297979df741cd560ccbea2ab4d572264bd703fc6cbc7ea2acb5c4/usr/lib" pid=30955 comm="ntpd" requested_mask="r" denied_mask="r" fsuid=0 ouid=0

Docker also assigns the container an address on the same subnet as my network. At first I though it was simply adding a line to the /etc/hosts file of the container. Which it does, but even after removing it issues are still seen.

Example:

root@myhostname:/# cat /etc/hosts  
X.X.192.3   myhostname.mydomain.com myhostname
127.0.0.1   localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
root@myhostname:/# ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:09:1a:c0:03  
          inet addr:X.X.X.X  Bcast:X.X.X.X  Mask:255.255.252.0
          inet6 addr: fe80::42:9ff:fe1a:c003/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:231 errors:0 dropped:26 overruns:0 frame:0
          TX packets:46 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:23719 (23.7 KB)  TX bytes:3960 (3.9 KB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

If no hostname is assigned via the command line (-h) to start the container, it will use the randomly assigned hostname

Environment:

Host:  
OS: Ubuntu 14.04.2 
Arch: PPC64LE
Kernel: 3.16.0-30
docker version
:
Client version: 1.4.1-dev
Client API version: 1.17
Go version (client): gccgo (GCC) 5.0.0 20150118 (experimental)
Git commit (client): 7294f26
OS/Arch (client): linux/ppc64le
Server version: 1.4.1-dev
Server API version: 1.17
Go version (server): gccgo (GCC) 5.0.0 20150118 (experimental)
Git commit (server): 7294f26

Hardware:

IBM Power 8, 8247-22L
CPUs: 192
Memory: 512GB

Containers:

OS: Ubuntu 14.04.2 
Arch: PPC64LE
Kernel: 3.16.0-30

Network:

brctl show 
bridge name bridge id       STP enabled interfaces
br0     8000.6cae8b6aaf64   no      eth0

br0       Link encap:Ethernet  HWaddr 6c:ae:8b:6a:af:64  
          inet addr:X.X.X.X  Bcast: X.X.X.X Mask:255.255.252.0
          inet6 addr: fe80::6eae:8bff:fe6a:af64/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6750867 errors:0 dropped:80 overruns:0 frame:0
          TX packets:1586308 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1049658934 (1.0 GB)  TX bytes:6936734104 (6.9 GB)

eth0      Link encap:Ethernet  HWaddr 6c:ae:8b:6a:af:64  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:7155242 errors:0 dropped:7598 overruns:0 frame:0
          TX packets:6347549 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1212526214 (1.2 GB)  TX bytes:7339350703 (7.3 GB)
          Interrupt:249 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:238 errors:0 dropped:0 overruns:0 frame:0
          TX packets:238 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:22564 (22.5 KB)  TX bytes:22564 (22.5 KB)
Federico Sierra
  • 3,589
  • 1
  • 20
  • 26
Brad
  • 41
  • 1
  • 4

3 Answers3

2

I also tried to solve this problem some time ago. I couldn't find a way to alter the behavior of docker regarding it's IP assignments without disabling the whole networking functionality (--net=none). So I assigned a so far unused subnet to the docker bridge interface and let docker do the assignment. I also wrote a daemon that listens for certain docker API events and then pushes dynamic updates to our DNS server, so I don't have to care about the IP's and can address the containers by name.

Here's the github repo for the DNS daemon. https://github.com/cschritt/docker-dns-daemon I will push my latest changes and some refactoring soon.

john.dough
  • 103
  • 1
  • 10
1

Unfortunately it seems there is no way to prevent Docker from assigning IP addresses to containers in bridge mode. As workaround I add pre-up ip addr flush dev eth0 to /etc/network/interface in order to dismiss Docker's IP address as described here. Here is an example how it can be done:

auto eth0
iface eth0 inet static
  pre-up ip addr flush dev eth0
  address 192.168.0.249
  netmask 255.255.255.0
  gateway 192.168.0.1

The caveat of this approach is with multiple containers when Docker injects its own incorrect/overridden IP addresses into containers' /etc/hosts files. The only way I've found to fix this is to install inotify-tools package and run the following script to revert /etc/hosts to its pre-defined state as soon as Docker changes it:

#!/bin/sh

while /usr/bin/inotifywait -e close /etc/hosts; do
  cp -v /etc/hosts.orig /etc/hosts
done

The above script requires you to add valid /etc/hosts.orig file into container.

Onlyjob
  • 348
  • 1
  • 7
0

Beginning with Docker 1.10, set to be released in the next few days, you can specify a static IP address explicitly when starting your container, with the --ip= and --ip6= options, to specify IPv4 and IPv6 addresses respectively. These can be used with docker run and docker network connect and persist as long as the container exists.

Michael Hampton
  • 244,070
  • 43
  • 506
  • 972