Questions tagged [kubernetes-jobs]
139 questions
2
votes
1 answer
Small pods starving big pods on an oversubscribed Kubernetes cluster
Our group has recently set up a 3-node Kubernetes cluster, and we've been using Jobs to schedule batch processing tasks on it. We have a lot of work to do and not a particularly large cluster to do it on, so at any given time there are a bunch of…
interfect
- 2,665
- 1
- 20
- 35
2
votes
2 answers
How to obtain Ip address of a kubernetes pod by querying DNS srv records?
I am trying to create a kubernetes job inside which I will run "dig srv" queries to find out the IP address of all the pods for any specific service running on the same cluster.
Is this achievable ?
I would like to elaborate a little more on the…
nakul shukla
- 138
- 1
- 8
2
votes
1 answer
Running jobs at most once in Kubernetes
In the kubernetes documentation there is the following note at the page https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/
Note that even if you specify .spec.parallelism = 1 and
.spec.completions = 1 and…
testTester
- 2,371
- 3
- 18
- 22
2
votes
3 answers
kubernetes jobs init container
I've created a JOB pod with 2 INIT containers
At pod creation, my job completed successfully but no sign of the init containers
To me, the job should have wait for the completion of the 2 init containers before starting
My job blueprint (taken from…
Paul Pozicheun
- 29
- 1
- 1
- 2
1
vote
1 answer
k8s job - keep a failed pod
Below is my definition for a k8s job (to convert a column of a mysql table from int->bigint using Percona's pt-online-schema-change):
apiVersion: batch/v1
kind: Job
metadata:
name: bigint-tablename-columnname
namespace: prod
spec:
…
callum
- 57
- 6
1
vote
1 answer
Kubernetes Jobs: How to manually end a job without killing current running pods?
I have a job which currently runs with "pararellism" spec set to 5. (5 running pods at each given moment) I want the job to stop spawning new pods after each fail, but let the current running pods end uninterrupted.
Tried to set pararellism to 1 in…
John Appleboim
- 44
- 4
1
vote
1 answer
K8s: Converting a completed job into a pod?
I have a job that runs on deployment of our app. The job runs fine 99.9% of the time but every so often something goes wrong (in the application config) and we need to run commands by hand. Because the job has several initContainers it's not as…
Mr Morphe
- 664
- 1
- 7
- 15
1
vote
1 answer
Kubernetes Job doesnt complete if the pod end up with error
we are doing case study with control-m to monitor Kubernetes job.
On successful completions of job, control -m is able to recognize the job completed.
however when it fails, it never recognize the failure it shows job is still running,i suspect job…
Programmer007
- 67
- 1
- 7
1
vote
1 answer
k8s job pod resource usage
For regular pods (in running state), we can check the actual resource utilisation (runtime) using kubectl top pod command.
However, for the job pods (execution is already complete), any way we can fetch how much resources were consumed by…
Kalyani
- 11
- 2
1
vote
0 answers
How to handle long running async task using celery in a python project
Current project has few periodic celery async tasks which takes more than 1 hour to finish by celery worker. Ideally these tasks should have been properly designed and broken down into smaller tasks but it is not possible for now. What should be the…
Nitish Chauhan
- 375
- 4
- 7
1
vote
1 answer
Google Cloud Function https certificate to connect to GKE
Due to the difficulty in using the official java kubernetes client inside a cloud function (basically we didn't know how to put the .kube/config file in the environment for the classes of the client to authenticate against) we have decided to try…
carlos palma
- 722
- 3
- 12
- 29
1
vote
1 answer
How to add env vars into k8s job.yaml dynamically?
I have store my env vars in aws secret manger.In CD pipeline I will read this env from secret manager and store into env.yaml once it is available in env.yaml,i want to render it to kubernetes job.yaml file. How to render env var from env.yaml to my…
Akshay Gopani
- 473
- 4
- 16
1
vote
3 answers
How can I delete all pods via delete job
I created a job,and rerun it several times。
When I delete this job. Only the latest pod be deleted.
How Can I delete these pods all.
Lu Peng
- 23
- 2
1
vote
1 answer
Jobs not getting cleaned up until 30+ minutes after TTL has passed
We are using the Job ttlSecondsAfterFinished attribute to automatically clean up finished jobs. When we had a very small number of jobs (10-50), the jobs (and their pods) would get cleaned up approximately 60 seconds after completion. However, now…
C_Z_
- 7,427
- 5
- 44
- 81
1
vote
1 answer
dockerfile and kubernetes jobs ( assistance needed)
I have my dockerfile in which i have used postgres:12 image and i modified it using some ddl scripts and then i build this image and i can run the container through docker run command but how i can use Kubernetes jobs to run build image , as I dont…
user17507960