What happens when kubernetes readiness-probe return false? How long to wait?

Question

What happens when Kubernetes readiness-probe returns false? Does Kubernetes restart that pod after timeout? How long Kubernetes waits for readiness?

Answer 1

Readiness probe doesn't restart the pod/container, readiness probe determines that the container is ready to serve traffic. If the container is probed and considered not "ready", the container will be removed from the endpoints and traffic wont be sent to it, until it is ready again.

[1] https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-readiness-probes

[2] https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#container-probes

[3] kubectl explain pod.spec.containers.readinessProbe

KIND:     Pod
VERSION:  v1

RESOURCE: readinessProbe <Object>

DESCRIPTION:
     Periodic probe of container service readiness. Container will be removed
     from service endpoints if the probe fails. Cannot be updated. More info:
     https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes

     Probe describes a health check to be performed against a container to
     determine whether it is alive or ready to receive traffic.

FIELDS:
   exec <Object>
     One and only one of the following should be specified. Exec specifies the
     action to take.

   failureThreshold <integer>
     Minimum consecutive failures for the probe to be considered failed after
     having succeeded. Defaults to 3. Minimum value is 1.

   httpGet  <Object>
     HTTPGet specifies the http request to perform.

   initialDelaySeconds  <integer>
     Number of seconds after the container has started before liveness probes
     are initiated. More info:
     https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes

   periodSeconds    <integer>
     How often (in seconds) to perform the probe. Default to 10 seconds. Minimum
     value is 1.

   successThreshold <integer>
     Minimum consecutive successes for the probe to be considered successful
     after having failed. Defaults to 1. Must be 1 for liveness and startup.
     Minimum value is 1.

   tcpSocket    <Object>
     TCPSocket specifies an action involving a TCP port. TCP hooks not yet
     supported

   timeoutSeconds   <integer>
     Number of seconds after which the probe times out. Defaults to 1 second.
     Minimum value is 1. More info:
     https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes

Let's use the default readiness probe from the documentation:

cat pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80
    readinessProbe:
      exec:
        command:
        - cat
        - /tmp/healthy
      initialDelaySeconds: 5
      periodSeconds: 5

To perform a probe, the kubelet executes the command cat /tmp/healthy in the target container. If the command succeeds, it returns 0, then the container is ready and can "serve". if the command returns anything but 0, container is not healthy.

Since this file doesn't exist in the container from the start, when the pod starts, it is going to be very unhealthy.

date && k get pods nginx
Thu  2 Dec 2021 19:08:43 AST
NAME    READY   STATUS    RESTARTS   AGE
nginx   0/1     Running   0          66s

now, lets exec into it and create the file, so that the command succeeds.

k exec -it nginx -- bash
root@nginx:/# touch /tmp/healthy
root@nginx:/# exit
exit

checking again:

date && k get pods nginx
Thu  2 Dec 2021 19:09:26 AST
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          110s

removing again:

k exec -it nginx -- bash
root@nginx:/# rm /tmp/healthy
root@nginx:/# exit
exit

checking:

date && k get pods nginx
Thu  2 Dec 2021 19:09:53 AST
NAME    READY   STATUS    RESTARTS   AGE
nginx   0/1     Running   0          2m17s