0

i do not 100% understand if SpringBatch, in a multi-instanz Kubernets application, work fine. I read Batch Processing on Kubernetes, so i understand that in general it works fine, but in the answer it is not mentioned that it works fine in a multi-instanz installation using ONE database.

Our setup looks like this: We have multiple instances of the application running in Kubernets and they sharing ONE database. Some jobs would be triggered by user interaction (in one of the many pod's that is answering the request from the UI) and some are triggerd by cronjob from kubernetes (eg data reorg) (in one of the many pod's that is answering the REST request from the cronjob). All pods are containing the incidental application.

Does this setup work fine with SpringBatch?

thanks for your help :-)

user286974
  • 111
  • 1
  • 1
  • 12
  • What do you mean by `in a multi-instanz Kubernets application using ONE database`? `We have multiple instances of the application running in Kubernets and they sharing ONE database.`: The linked answer explains how a centralized job repository, ie shared by all jobs running on different machines, allows you to prevent duplicate job executions, save job state and restart from the last save point in case of failure, etc. So please elaborate on what do you mean by `multi-instanz Kubernets application` with a concrete example. – Mahmoud Ben Hassine Feb 17 '21 at 09:41
  • Moreover, the linked blog post uses a single database shared by all jobs running on k8s: https://spring.io/blog/2021/01/27/spring-batch-on-kubernetes-efficient-batch-processing-at-scale#3-deploy-the-job-on-kubernetes. Is this what you are looking for? – Mahmoud Ben Hassine Feb 17 '21 at 09:42
  • Hi Mahmoud, with `multi-instanz Kubernets application` i mean `replicas` in the helm chart has a value bigger than one. I was not so sure that the linked article really reference the scenario with `replicas`>1. – user286974 Feb 17 '21 at 14:55

1 Answers1

1

As far as Spring Batch is concerned, all these things are deployment details. It is up to you to design your jobs and job instances with that in mind. This is what I explained in details in the Choosing the right Spring Batch job parameters and Choosing the Right Kubernetes Job Deployment Pattern sections. Note that this blog post is linked in the answer you shared.

What Spring Batch guarantees, thanks to the centralized job repository design (which is what you are referring to as "ONE database"), is preventing duplicate and concurrent job executions of the same job instance.

So the answer to your question is yes, as long as you choose the right deployment pattern for your Spring Batch jobs and Kubernetes jobs.

Mahmoud Ben Hassine
  • 28,519
  • 3
  • 32
  • 50