0

My spring batch application is running on PCF platform which is connected to MySQL database (single instance), it's running fine when only an instance is up & running but when it comes to more than one application instance, I'm getting exception org.springframework.dao.DuplicateKeyException. This might be happening because similar batch job is firing at the same time & trying to update batch instance table with same job ID. Is there any way to restrict this kind of failure or in another way, I wanted a solution where only one batch job will run at a time even there are multiple instances running.

JoshMc
  • 10,239
  • 2
  • 19
  • 38
Surajit Sarkar
  • 111
  • 1
  • 3
  • Probably this can be a solution https://stackoverflow.com/questions/23193779/spring-batch-restrict-single-instance-of-job-only – Mohit Sharma Feb 28 '21 at 17:51
  • Please post the full stack trace of the error. Is your issue similar to https://stackoverflow.com/questions/14321544/sequences-not-working-when-switching-a-spring-batch-job-repository-database-to-m? – Mahmoud Ben Hassine Mar 01 '21 at 13:05

1 Answers1

0

For me , it is a good sign that DuplicateKeyException is thrown. Because it exactly achieves what you want to do is that spring-batch already makes sure that the same job execution will not executed in parallel. (i.e. Only one server instance execute the job successfully while other fail to execute)

So I see no harms in your case. If you don't like this exception , you can catch it and re-throw it as your application level exception saying something like "The job is executing by other sever instances , so skip to execute it."

If you really want that only one server instance will try to trigger to execute a job and other servers will not try to trigger in the meantime , it is not the problem of spring-batch but is a problem about how you ensure that only one server node will fires the request in the distributed environment. If the batch job is fired as a scheduled task using @Scheduled , you can consider to use a distributed lock such as ShedLock to make sure that it is executed at most once at the same time on one node only.

Ken Chan
  • 84,777
  • 26
  • 143
  • 172
  • Thanks for your suggestion about ShedLock. I will try to implement that one in my project since it looks very relevant for my case.. – Surajit Sarkar Mar 02 '21 at 07:05
  • @SurajitSarkar ShedLock uses a central db to ensure the same task is not executed on different nodes. This is the same technique used already in Spring Batch to prevent running the same job instance on different nodes. So you can have that in Spring Batch by correctly choosing the identifying job parameters (which will define your job instances). I wrote about this in details here: https://spring.io/blog/2021/01/27/spring-batch-on-kubernetes-efficient-batch-processing-at-scale#2-choosing-the-right-spring-batch-job-parameters – Mahmoud Ben Hassine Mar 02 '21 at 11:04