7

Docker creates a veth interface connected to a bridge (docker0) for each of the containers it create.

http://docs.docker.io/use/networking/

I want to limit the bandwidth these new veth interfaces have. I found a way to do this with wondershaper. However I want to automate this.

Is there a way to have a hook that runs a script every time a new veth interface is attached?

I have looked into adding scripts in /etc/network/if-up.d/, but they do not run when a veth is added only during boot.

Here are some syslogs of what I am trying to get notified about. I know I can tail these logs but that method seems sort of hacky and there has to be a way to get notified about this event via the OS.

May  2 23:28:41 ip-10-171-7-2 kernel: [22170163.565812] netlink: 1 bytes leftover after parsing attributes.
May  2 23:28:42 ip-10-171-7-2 kernel: [22170163.720571] IPv6: ADDRCONF(NETDEV_UP): veth5964: link is not ready
May  2 23:28:42 ip-10-171-7-2 kernel: [22170163.720587] device veth5964 entered promiscuous mode
May  2 23:28:42 ip-10-171-7-2 avahi-daemon[1006]: Withdrawing workstation service for vethdc8c.
May  2 23:28:42 ip-10-171-7-2 kernel: [22170163.743283] IPv6: ADDRCONF(NETDEV_CHANGE): veth5964: link becomes ready
May  2 23:28:42 ip-10-171-7-2 kernel: [22170163.743344] docker0: port 27(veth5964) entered forwarding state
May  2 23:28:42 ip-10-171-7-2 kernel: [22170163.743358] docker0: port 27(veth5964) entered forwarding state
May  2 23:28:48 ip-10-171-7-2 kernel: [22170170.518670] docker0: port 26(vethb06a) entered forwarding state
May  2 23:28:57 ip-10-171-7-2 kernel: [22170178.774676] docker0: port 27(veth5964) entered forwarding state
errordeveloper
  • 6,716
  • 6
  • 41
  • 54
AnandKumar Patel
  • 980
  • 9
  • 18

2 Answers2

3

You should write a custom udev rule that runs a script of yours each time a new interface is added. This is what Debian does for handling interface "hotplug".

/etc/udev/rules.d/90-my-networking.rules:

SUBSYSTEM=="net",           RUN+="/usr/local/bin/my-networking-agent.sh"

/usr/local/bin/my-networking-agent.sh:

#!/bin/sh
logger "hey I just got interface ${INTERFACE} with action ${ACTION}"

EDIT

Here is how you can test it:

# modprobe dummy0
# ifconfig dummy0 up
# tail -n1 /var/log/syslog
May  3 01:48:06 ernst logger: hey I just got interface dummy0 with action add
Grapsus
  • 2,714
  • 15
  • 14
1

The udev rules are one of the ways to do it, however there is some shortage of information there, i.e. there is no reliable and simple way to know which container the veth is associated with. I am not sure whether in your case it's sufficient to simply set bandwidth limit on the hosts' end of veth pair, which might be the cease, but there is also the other end of it in the container's namespace, which is something you can look at using either ip netns or nsenter commands. So if you do need to operate on both ends of the veth pair, it's best to have container ID, so you can lookup the PID and the network namespace associated with it. One way to do it is by running docker events and parsing it's output, and yet a better way is to use Docker's domain socket API. For a use case I had earlier, it was sufficient to shell-out to docker events and here is a script I wrote, what it does is add a route inside a container and turn-off the checksum offload with ethtool.

#!/bin/sh -x

[ ! $# = 2 ] && exit 1;

container_interface="$1"
add_route="$2"

docker events | while read event
do
    echo $event | grep -q -v '\ start$' && continue

    container_id=`echo $event | sed 's/.*Z\ \(.*\):\ .*/\1/'`

    nsenter="nsenter -n -t {{ .State.Pid }} --"
    ip_route_add="ip route add ${add_route} dev ${container_interface} proto kernel scope link src {{ .NetworkSettings.IPAddress }}"
        ethtool_tx_off="ethtool -K ${container_interface} tx off >/dev/null"

    eval `docker inspect --format="${nsenter} ${ip_route_add}; ${nsenter} ${ethtool_tx_off};" ${container_id}`
done

In addition to docker events, there is another way of catching networking events with ip monitor command. However, that way you still don't have container IDs, similarly to udev method.

errordeveloper
  • 6,716
  • 6
  • 41
  • 54