1

I have a deployment of my backend app which has 6 replicas. In the code of the app, I have a node.js cronjob running every minute. But the problem is that because I have 6 replicas, it runs 6 times in parallel. I want to send env variable to exactly only one Pod in the deployment to ensure that only 1 Pod performs the cronjob. Then in the app code I will check this using the process.env.SHOULD_PROCESS_CRONJOBS == 1.

If this is not the right way to achieve what I need, can you inform me on how to handle cronjobs using nodejs in distributed environment like K8S?

Raz Buchnik
  • 7,753
  • 14
  • 53
  • 96
  • 1
    Are you using regular cron in the pod? or some form of node scheduler? k8s has the ability to schedule a [CronJob](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/) outside of your application pods. – Matt Feb 05 '22 at 03:44
  • The cronjob itself should be run separately outside of the app in its own container / job instance. Look into decoupling your cron logic from your application so that it can run independently on its own as a [CronJob](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/). – maiorano84 Feb 05 '22 at 03:45
  • If node.js is doing the scheduling, The jobs container template would be largely the same as the applications. Just change the `command` to the short lived task, instead of the long lived application. – Matt Feb 05 '22 at 03:48
  • The k8s is doing the scheduling. So basically you say that I need to change the code - removing the cronjob from the app and create another app just for this cronjob that will be run using the k8s cronjob resource? – Raz Buchnik Feb 05 '22 at 04:03
  • Raz Buchnik, @Matt and maiorano84 proposed to schedule a job in k8s using a CronJob (a built-in feature of Kubernetes). It is the best way to schedule jobs in Kubernetes. Do you think the proposed CronJob is an acceptable answer to your initial question? – mozello Feb 07 '22 at 17:32

1 Answers1

1

Let Kubernetes schedule a CronJob in addition to the application

apiVersion: batch/v1
kind: CronJob
metadata:
  name: app-cron
spec:
  schedule: "* * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: my-app-cron
            image: same-as-app
            imagePullPolicy: IfNotPresent
            command:
            - /bin/sh
            - -c
            - date; echo Hello from scheduled job
          restartPolicy: OnFailure

If you don't want kubernetes supervising the cron scheduling, then another option is to split out the application that schedules the cron into two Deployments. Only use 1 replica for the container that has environment variable:

      env:
      - name: SHOULD_PROCESS_CRONJOBS
        value: "1"
Matt
  • 68,711
  • 7
  • 155
  • 158