0

I'm looking at a Docker swarm host, which is the only host in our Docker swarm cluster (no, not recommended).

Due to a DNS issue in Moby we've sent intra-container traffic via either the host's external IP (192.168.1.50) or via docker_gwbridge (172.18.0.1). This works as these requests will be DNAT'ed to 172.18.0.2 (which fronts a Docker container handling the VIP / load balancing functionality)

We see randomly that request duration between an nginx container and memcached container (two different containers) takes 1 second or more.

We see this by running mc_conn_tester.pl inside the nginx container.

  • When traffic between the two containers go via 192.168.1.50, the mc_conn_tester.pl reports a lot of 1 sec requests when we use 192.168.1.50 also in the test. If we run mc_conn_tester.pl against 172.18.0.1 instead, we do not see any 1 sec requests.

  • When traffic between the two containers go via 172.18.0.1, the mc_conn_tester.pl reports a lof of one 1 sec requests when we use 172.180.1 in the test. If we run mc_conn_tester.pl against 192.168.1.50 instead, we do not see any 1 sec requests.

What could cause this behaviour and give 1 sec (and up) delays, and how can we debug this further?

Followup on https://github.com/moby/moby/issues/36032

sbrattla
  • 1,578
  • 4
  • 28
  • 52
  • could you create an image and info for others to test this setup? I'd say before t/s further, be sure you're on a 4.x kernel, using 17.12 latest stable release, and don't use CPU-cycle limiting VM's like AWS T2. – Bret Fisher Jan 16 '18 at 06:22
  • @BretFisher we upgraded to 17.12 this morning, so we'll be keeping an eye. Running on Hyper-V and 4.x kernel. We discovered in the end that the ingress-sbox drops a lot of packets from the nginx, which memcached never gets. – sbrattla Jan 16 '18 at 06:57

0 Answers0