Dynamic Command for Kubernetes Jobs

Question

So hopefully this makes sense to the non-Djangoers of the k8s community. I will try my best to explain the setup / reasoning.

With Django, we have LOTS of what are called management commands that we can run within the scope and environment of our Django app that can really help development and deployment. I'm sure most other frameworks have similar, if not identical, concepts.

An example would be the "python manage.py migrate" command that ensures our codebase (migration scripts) are applied to and reflect in the associated database.

There are approx. 30 - 50 core commands we can run, we can also create our own, as well as apply those from any installed third party applications.

Anyways. The most important takeaway is that there are a lot of commands we can and do run.

Now, I have the following k8s Job to run the "migrate" command:

apiVersion: batch/v1
kind: Job
metadata:
  name: asencis-web-migrate-job
spec:
  template:
    spec:
      containers:
        - name: asencis-web-migrate-job
          image: asencis/asencis-base:latest
          imagePullPolicy: Always
          command: ['python', 'manage.py', 'migrate']
          envFrom:
          - configMapRef:
              name: asencis-config
          - secretRef:
              name: asencis-secret
      restartPolicy: Never
  backoffLimit: 5

This job essentially runs the python manage.py migrate command within the application scope/environment. It works like a charm:

$ kubectl apply -f asencis-web-migrate-job.yaml

It's very useful in the deployment of the application when all of our tests have run, we can then build an image, "rollout restart" the cluster and then apply any migrations. It's incredibly seamless. (Not of my hat to the k8s core team for making such a useful product!)

Anyways.

My question is essentially this, can we apply an argument to the above kubectl apply command on the job, to run any command we like?

An example would be:

$ kubectl apply -f asencis-web-job.yaml --command python manage.py migrate

Answer 1

You will probably need to build your own tool for this.

You could use a shell script around yq, for example:

#!/bin/sh
yq eval \
    ".spec.template.spec.containers.[0].command.[2]=\"$1\"' \
    template-web-job.yaml \
| kubectl apply -f-

You can fill in more parts of the Job this way: compute .metadata.name from $USER-$1-$(date +%s), attach labels: to the Pod to find it later, and so on.

If this wasn't a one-off Job I might recommend a more Kubernetes-native tool like Helm or Kustomize. Both tools need somewhat involved filesystem layouts, and then you need to pass the variables (script command, submitter) in some form; this wouldn't actually be easier than using yq to rewrite the YAML. Helm is a little more oriented towards mostly-stable installations (it knows about the main application Deployment and how to update it in-place).

If you have Helm already, you could build a similar script around helm template. Or if you have jq but not yq you could rewrite the Job in JSON {"apiVersion": "batch/v1", "kind": "Job", ...} but otherwise use the same script.