So far I thought I had a good understanding of Kubernetes. Today I was startled to see a service forwarding traffic to an unready pod. Can somebody fill this knowledge gap for me? Here is what I'm doing:
I'm using the latest version of Kind to spin up a Kubernetes cluster (Version: 1.25). On the cluster I'm running a deployment configured to run 3 replicas of a pod running a HTTP server. Next to it is a Service of type NodePort forwarding traffic to the 3 pods. I have configured Kind for use with NodePort services and it works, meaning I can access the web server from outside the cluster.
One of my HTTP requests deliberately crashes a process in one of the pods resuting in this pod's readiness probe to fail while the pod keeps running.
- I can see the status of the pod becoming unready.
- When describing the service, I can see that the unready pod was removed from the list of endpoints of that service.
BUT: Making a request to the node port, I get a response from the unready port. Why? I was expecting the service to forward requests only to ready pods...
I have tried using port-forwarding to the same service. Same result. Is this expected bevaviour of services of type node port?
I can paste my manifest as a bunch of yaml blobs but maybe this is aready clear to someone out there?
Thanks!
