Finished pods created by Kubernetes CronJob getting deleted after sometime

Question

I created a CronJob by using below yaml file.

kind: CronJob
metadata:
  name: $DEPLOY_NAME
spec:
  # Run the job once a day at 8 PM
  schedule: "0 20 * * *"
  # If the previous job is not yet complete during the scheduled time, do not start the next job
  concurrencyPolicy: Forbid
  jobTemplate:
      spec:
        # The pods will be available for 3 days (259200 seconds) so that logs can be checked in case of any failures
        ttlSecondsAfterFinished: 259200
        template:
          spec:
            containers:
            - name: $DEPLOY_NAME
              image: giantswarm/tiny-tools
              imagePullPolicy: IfNotPresent
              resources:
                requests:
                  cpu: "0.01"
                  memory: 256Mi
                limits:
                  cpu: "0.5"
                  memory: 512Mi
              command: ["/bin/sh"]
              args: ["-c", "cd /home/tapi && sh entrypoint.sh"]

As mentioned in ttlSecondsAfterFinished, k8s keeps my job in the cluster. However, the pod created by the job (after completion) gets deleted after sometime.

As per garbage collection policy, my pod object should be dependent upon my job. And since job object is not garbage collected, my pod object should also remain alive. Am i missing something?

Not very sure. 2-3 hours may be. This job is executed at night. And in morning, when we try to get the logs from pod, pod itself is not available. We tried scheduling job in day, at that time pod remained for atleast 2 hours. — Manish Bansal, Nov 15 '19 at 16:08
Check the Kubernetes version - TTL for finished jobs by setting `.spec.ttlSecondsAfterFinished` was introduced in v1.12. If the version is v1.12+ then check if the feature gate `TTLAfterFinished` is enabled. — apisim, Nov 15 '19 at 16:39
@apisim version is 1.15. also feature gate must be enabled as my job objects remain alive for 3 days as configured. — Manish Bansal, Nov 15 '19 at 17:17
Can you check `kubectl get jobs -o yaml` to see if TTL is there? — Crou, Nov 18 '19 at 10:42

score 1 · Answer 1 · edited Dec 06 '22 at 12:32

1

The .spec.successfulJobsHistoryLimit and .spec.failedJobsHistoryLimit fields are optional. These fields specify how many completed and failed jobs should be kept. By default, they are set to 3 and 1 respectively.

You might need to set these fields to appropriate value

edited Dec 06 '22 at 12:32

Dherik

17,757
11
115
164

answered Nov 15 '19 at 17:12

P Ekambaram

15,499
7
34
59

Although, when i check `kubectl get jobs`, i only see 3 jobs. But if i check ` kubectl describe cronjob abc`, failedjobhistortlimit is coming as 1. But success limit is coming as very high number as 824638391724 – Manish Bansal Nov 15 '19 at 17:15
You need to describe the job that is completed – P Ekambaram Nov 15 '19 at 17:53
That also i did. But there is no events in that. Please tell me what else should i check in that. In job,. There is no success and failure limits defined. – Manish Bansal Nov 16 '19 at 02:10

Finished pods created by Kubernetes CronJob getting deleted after sometime

1 Answers1