4

I couldn't find anything in the official kubernetes docs about this. What's the actual low-level process for replacing a long-running cron job? I'd like to understand this so my application can handle it properly.

  • Is it a clean SIGHUP/SIGTERM signal that gets sent to the app that's running?
  • Is there a waiting period after that signal gets sent, so the app has time to cleanup/shutdown before it potentially gets killed? If so, what's that timeout in seconds? Or does it wait forever?

For reference, here's the Replace policy explanation in the docs:

https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/

Concurrency Policy

  • Replace: If it is time for a new job run and the previous job run hasn’t finished yet, the cron job replaces the currently running job run with a new job run
Community
  • 1
  • 1
mjuarez
  • 16,372
  • 11
  • 56
  • 73

1 Answers1

6

A CronJob has just another Pod underneath.

When a Cronjob with a concurrency policy of "Replace" is still active, the Job will be deleted, which also deletes the Pod.

When a Pod is deleted the Linux container/s will be sent a SIGTERM and then a SIGKILL after a grace period, defaulted to 30 seconds. The terminationGracePeriodSeconds property in a PodSpec can be set to override that default.

Due to the flag added to the DeleteJob call, it sounds like this delete is only deleting values from the kube key/value store. Which would mean the new Job/Pod could be created while the current Job/Pod is still terminating. You could confirm with a Job that doesn't respect a SIGTERM and has a terminationGracePeriodSeconds set to a few times your clusters scheduling speed.

Matt
  • 68,711
  • 7
  • 155
  • 158