Kubernetes cronjob that calls all pods in a service

Question

i have a netcore webapi deployed on kubernetes. Every night at midnight i need to call an endpoint to do some operations on every pod, so i have deployed a cronjob that calls the api with curl and the method does the required operations.

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: test-cronjob
spec:
  schedule: "0 0 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: test-cronjob
            image: curlimages/curl:7.74.0
            imagePullPolicy: IfNotPresent
            command:
              - "/bin/sh"
              - "-ec"
              - |
                date;
                echo "doingOperation"
                curl POST "serviceName/DailyTask"
          restartPolicy: OnFailurey

But this only calls one pod, the one assigned by my ingress. There is a way to call every pod contained in a service?

score 2 · Answer 1 · answered Feb 12 '21 at 08:38

That is an expected behavior as when we do curl on a Kubernetes Service object, it is expected to pass the requests to only one of the endpoints (IP of the pods). To achieve, what you need, you need to write a custom script that first gets the endpoints associated with the services and then iteratively call curl over them one by one.

Note: The pods IP can be changed due to pod re-creation so you should fetch the endpoints associated with the service in each run of the cronjob.

score 1 · Answer 2 · answered Feb 12 '21 at 08:39

You could run kubectl inside your job:

kubectl get pods -l mylabel=mylabelvalue \
    -o go-template='{{range .items}}{{.status.podIP}}{{"\n"}}{{end}}'

This will return the internal IP of all the containers my the specific label. You can then loop over the addresses and run your command.

score 0 · Answer 3 · answered Feb 12 '21 at 10:17

Since enabling Pods with rights on the API-Server, e.g. to look at the actual service endpoints, is cumbersome and poses a security risk, I recommend a simple scripting solution here.

First up, install a headless service for the deployment in question (a service with clusterIP=None). This will make your internal DNS Server create several A records, each pointing at one of your Pod IPs.

Secondly, in order to ping each Pod in a round-robin fashion from your Cron-Job, employ a little shell script along the lines below (your will need to run this from a container with dig and curl installed on it):

dig +noall +answer <name-of-headless-service>.<namespace>.svc.cluster.local | awk -F$'\t' '{curl="curl <your-protocol>://"$2":<your-port>/<your-endpoint>"; print curl}' | source /dev/stdin

Kubernetes cronjob that calls all pods in a service

3 Answers3