2

I have a preStop hook for my pod that sends TSTP signals to each process, and waits for their completion, but only to a maximum of 60 seconds.

Let us say that we set terminationGracePeriodSeconds to 3600 seconds. After the completion of the preStop hook, shouldn't it now send the TERM signal to my processes, and wait until the expiry of terminationGracePeriodSeconds before finally sending the KILL signal?

Because now it seems that after the completion of the preStop hook, it immediately sends a KILL signal and my pod is deleted abruptly, even when there is still a process running.

overflow_warrior_27
  • 93
  • 5

1 Answers1

3

No, it won't wait for the terminationGracePeriodSeconds.

This is the intended behavior for the preStop hook to send a SIGTERM on completion.

In case the preStop hook isn't finished after terminationGracePeriodSeconds countdown, kubelet will request 2 extra seconds for it to complete and then will send a SIGKILL to force the pod to shutdown.

The full sequence is :

  1. pod deletion has been requested
  2. preStop hook kicks in and terminationGracePeriodSeconds countdown starts :
    • if preStop hook completes, it sends a SIGTERM which stops the pods
    • if preStop hook isn't finished within terminationGracePeriodSeconds countdown, kubelet request 2 extra seconds before sending the SIGKILL which will stops the pod.

Sources :

Will
  • 1,792
  • 2
  • 23
  • 44
  • For more context, these are sidekiq workers(ruby background processing) that I am sending TSTP signals to, and waiting for X amount of time in the preStop hook. So let us say that the sidekiq jobs are still in the middle of running when the preStop hook finishes execution. So what is likely happening, is that my kubernetes pod responds to the SIGTERM by terminating immediately? But my understanding is that SIGTERM is a more "graceful" termination, and there are still sidekiq jobs running at the time that the pod is terminated. Shouldn't it wait for the terminationGracePeriod to expire? – overflow_warrior_27 Aug 29 '23 at 03:36
  • What about stoping those sidekiq processes part of the `preStop` hook ? – Will Aug 29 '23 at 07:13