DuplicateKey Exception while trying to run the batch job on multiple Instances

Question

I am running a batch job in a cluster at the same time with the same parameters. Though, its fine that it is running only on one instance but the exception I am getting is:

Detail: Key (job_name, job_key)=(offlineTicketRefreshJob, c5d36835a13fd8ae0e91a69a6fa1c2d8) already exists.; nested exception is org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "job_inst_un"

I was expecting that it will give JobAlreadyRunningException or the other. I think the isolation level is also serializable for batch job repository so why is it giving PSQLException?

score 1 · Accepted Answer · answered Mar 30 '20 at 11:23

1

That should not happen if you correctly configure the isolation level on your job repository. You did not share your job repository configuration but you can find an example int the Configuring a JobRepository and Transaction Configuration for the JobRepository sections:

// This would reside in your BatchConfigurer implementation
@Override
protected JobRepository createJobRepository() throws Exception {
   JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
   factory.setDataSource(dataSource);
   factory.setTransactionManager(transactionManager);
   factory.setIsolationLevelForCreate("ISOLATION_SERIALIZABLE");
   factory.setTablePrefix("BATCH_");
   factory.setMaxVarCharLength(1000);
   return factory.getObject();
}

An aggressive isolation level prevents creating duplicate job instances in a clustered environment. SERIALIZABLE and READ_COMMITTED should work.

answered Mar 30 '20 at 11:23

Mahmoud Ben Hassine

28,519
3
32
50

Isn't the default isolation is SERIALIZABLE as stated in the same documentation so I didnt configure the jobrepository and used the default. – sachin Mar 30 '20 at 13:08
yes, `SERIALIZABLE` is the default and postgres supports it (https://www.postgresql.org/docs/9.5/transaction-iso.html). Which version of postgres are yo using? – Mahmoud Ben Hassine Mar 30 '20 at 19:28
I am using 12.0 – sachin Mar 31 '20 at 04:17
Debugged the code, looks like it is setting ISOLATION_DEFAULT as isolation level which is read_committed in postgres. Tried configuring jobrepository as mentioned but it is giving issues as I am using JPAItemWriter for writing items which says "TransactionRequiredException", no transaction in progress. – sachin Mar 31 '20 at 12:36
The default is SERIALIZABLE: https://github.com/spring-projects/spring-batch/blob/d8fc58338d3b059b67b5f777adc132d2564d7402/spring-batch-core/src/main/java/org/springframework/batch/core/repository/support/AbstractJobRepositoryFactoryBean.java#L62. You should be overriding it somewhere if you have ISOLATION_DEFAULT when debugging. Now that's fixed, for JPAItemWriter to work, you need to make sure Spring Batch is configured with a JpaTransactionManager, see https://stackoverflow.com/questions/22509529 – Mahmoud Ben Hassine Mar 31 '20 at 12:49
One more question, If I don't configure the repository. Then there is no TransactionRequiredException. It should have been same thing for that.. right? – sachin Mar 31 '20 at 14:38
I am getting TransactionRequired at other places now as well.. apart from batch – sachin Apr 02 '20 at 09:37

DuplicateKey Exception while trying to run the batch job on multiple Instances

1 Answers1