Can't resolve another service's hostname inside my Kubernetes Windows setup

Question

I have recently followed the tutorial on how to use Kubernetes with Windows pods ( https://learn.microsoft.com/en-us/azure/container-service/kubernetes/container-service-kubernetes-windows-walkthrough ). I decided to extend the example to two services, one front calling the one in the back. Simplified:

https://gist.github.com/sebug/f478f1cfd0a793e8d556c6001bbbe142

But now when I connect to one of the front nodes:

kubectl exec -it samplefront-2836659004-4m824 -- powershell

I can't ping the other service:

PS C:\> ping sample-back
Ping request could not find host sample-back. Please check the name and try again.

I heard that it may be because of the two network interfaces and the wrong DNS server being chosen, but I have not found a way to specify anything in the deployment.

Windows IP Configuration


Ethernet adapter vEthernet (Container NIC 7baf5cc0):

Connection-specific DNS Suffix  . :
Link-local IPv6 Address . . . . . : fe80::f182:e2e7:7bce:ed60%33
IPv4 Address. . . . . . . . . . . : 10.244.0.211
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 10.244.0.1

Ethernet adapter vEthernet (Container NIC ae765bad):

Connection-specific DNS Suffix  . : 10jheu23yh0ujpey5vzw0q45qg.ax.internal.cloudapp.net
Link-local IPv6 Address . . . . . : fe80::c4dc:b785:9cd:2a7b%37
IPv4 Address. . . . . . . . . . . : 172.31.245.122
Subnet Mask . . . . . . . . . . . : 255.255.240.0
Default Gateway . . . . . . . . . : 172.31.240.1

Answer 1

Can't resolve another service's hostname inside my Kubernetes.

This is a by design behavior. Because the cluster IP does not exist.

In Kubernetes, all the services in a cluster are handled by kube-proxy. kube-proxy runs on every node in the cluster, and what it does it write iptables rules for each service (Linux node, same as windows). These iptables rules manage the traffic towards the service IPs. They don’t actually have any rules for ICMP, because it’s not needed.

But we can ping pod IP or pod's DNS.

For example, we can use this command to list pods IP addresses:

root@k8s-master-9F42C511-0:~# kubectl get pods -o wide
NAME                               READY     STATUS    RESTARTS   AGE       IP           NODE
azure-vote-back-3048739398-8zx8b   1/1       Running   0          18m       10.244.1.2   k8s-agent-9f42c511-0
azure-vote-front-837696400-tglpn   1/1       Running   0          18m       10.244.1.3   k8s-agent-9f42c511-0

Then we use one pod to ping those IP addresses:

root@k8s-master-9F42C511-0:~# kubectl exec -it azure-vote-front-837696400-tglpn -- /bin/bash
root@azure-vote-front-837696400-tglpn:/app# ping 10.244.1.3
PING 10.244.1.3 (10.244.1.3): 56 data bytes
64 bytes from 10.244.1.3: icmp_seq=0 ttl=64 time=0.063 ms
64 bytes from 10.244.1.3: icmp_seq=1 ttl=64 time=0.052 ms
^C--- 10.244.1.3 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.052/0.057/0.063/0.000 ms
root@azure-vote-front-837696400-tglpn:/app# ping 10.244.1.4
PING 10.244.1.4 (10.244.1.4): 56 data bytes
64 bytes from 10.244.1.4: icmp_seq=0 ttl=64 time=0.102 ms
64 bytes from 10.244.1.4: icmp_seq=1 ttl=64 time=0.098 ms
^C--- 10.244.1.4 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.098/0.100/0.102/0.000 ms

Also, we can ping pod's A record. In kubernetes, pod's A record in the form of pod-ip-address.my-namespace.pod.cluster.local.

For example, a pod with IP 1.2.3.4 in the namespace default with a DNS name of cluster.local would have an entry: 1-2-3-4.default.pod.cluster.local

In my lab, my pod's A record like this:

root@k8s-master-9F42C511-0:~# kubectl exec -it azure-vote-front-837696400-tglpn -- /bin/bash
root@azure-vote-front-837696400-tglpn:/app# ping 10-244-1-2.default.pod.cluster.local                                                                                                              
PING 10-244-1-2.default.pod.cluster.local (10.244.1.2): 56 data bytes
64 bytes from 10.244.1.2: icmp_seq=0 ttl=64 time=0.103 ms
64 bytes from 10.244.1.2: icmp_seq=1 ttl=64 time=0.087 ms
64 bytes from 10.244.1.2: icmp_seq=2 ttl=64 time=0.096 ms
^C--- 10-244-1-2.default.pod.cluster.local ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.087/0.095/0.103/0.000 ms

So, we can't ping cluster IP address, but we can use URL to test it. We can ping pod's IP address, and A record.

---

Update:
Sorry for my mistake, the k8s A record rules work for Linux Agent, but does not work for windows agent.

More information about windows server containers, please refer to this article.

Answer 2

So after trying out different scenarios I figured I'd delete the setup and try again, specifying a specific version of microsoft/iis - and it worked:

https://gist.github.com/sebug/0f7776668fff4e0e6b3f3d313846afa6

kripke:Documents/Projets/ScaledSample% kubectl exec -it samplefront-1226573881-21bbh -- ping sample-back


Pinging sample-back [10.0.216.120] with 32 bytes of data:
Reply from 10.0.216.120: bytes=32 time<1ms TTL=128
Reply from 10.0.216.120: bytes=32 time<1ms TTL=128
Reply from 10.0.216.120: bytes=32 time<1ms TTL=128
Reply from 10.0.216.120: bytes=32 time<1ms TTL=128

Ping statistics for 10.0.216.120:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms

My hypothesis is that either I fell on a more well-behaved instance after having recreated the resource group and cluster or that this specifying which exact version of the container image did the trick.