Commit failed while step execution data was already updated. Reverting to old version. Spring batch

Question

I was having problem starting up spring batch application, which I mentioned here. Now I have update the code to use HibernateTransactionManager and my job has 6 steps, and first step works fine now, but second steps fails. Following is update code in which I have added HibernateTransactionManager.

BatchScheduler.Class

import org.springframework.batch.core.launch.support.SimpleJobLauncher;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean;
import org.springframework.batch.support.transaction.ResourcelessTransactionManager;
import org.springframework.context.annotation.Bean;
import org.springframework.scheduling.annotation.EnableScheduling;

//@Configuration
@EnableScheduling
public class BatchScheduler {

  @Bean
  public ResourcelessTransactionManager transactionManager() {
    return new ResourcelessTransactionManager();
  }

  @Bean
  public MapJobRepositoryFactoryBean  mapJobRepositoryFactory(
      ResourcelessTransactionManager txManager) throws Exception {

    MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(txManager);
    factory.afterPropertiesSet();
    return factory;
  }

  @Bean
  public JobRepository jobRepository(MapJobRepositoryFactoryBean factory) throws Exception {
    return factory.getObject();
  }

  @Bean
  public SimpleJobLauncher jobLauncher(JobRepository jobRepository) {
    SimpleJobLauncher launcher = new SimpleJobLauncher();
    launcher.setJobRepository(jobRepository);
    return launcher;
  }
}

@Configuration
@EnableBatchProcessing
@Import({BatchScheduler.class})
public class LogBatchConfiguration {

  private static final Logger log = LoggerFactory.getLogger(LogBatchConfiguration.class);

  @Autowired private SimpleJobLauncher jobLauncher;

//**********New added
   @Bean
    public PlatformTransactionManager mytransactionManager() {
    HibernateTransactionManager transactionManager = new HibernateTransactionManager();
    transactionManager.setSessionFactory(createLocalSessionFactory(mainDataSource).getObject());
    return transactionManager;
    }


  @Autowired
  @Qualifier(value = "mainDataSource")
  public DataSource mainDataSource;

  @Autowired
  @Qualifier(value = "toolsDataSource")
  public DataSource toolsDataSource;

  @Autowired public StepBuilderFactory stepBuilderFactory;

  @Autowired public JobBuilderFactory jobBuilderFactory;

  @Autowired private RestTemplate restTemplate;

  @Qualifier(value = "createLocalSessionFactory")
  @Autowired
  private SessionFactory createLocalSessionFactory;

  @Qualifier(value = "createToolSessionFactory")
  @Autowired
  private SessionFactory createToolSessionFactory;

  @Bean
  RestTemplate restTemplate(RestTemplateBuilder builder) {
    RestTemplate restTemplate = new RestTemplate();
    List<HttpMessageConverter<?>> converters = restTemplate.getMessageConverters();
    for (HttpMessageConverter<?> converter : converters) {
      if (converter instanceof MappingJackson2HttpMessageConverter) {
        MappingJackson2HttpMessageConverter jsonConverter =
            (MappingJackson2HttpMessageConverter) converter;
        jsonConverter.setObjectMapper(new ObjectMapper());
        jsonConverter.setSupportedMediaTypes(
            ImmutableList.of(
                new MediaType(
                    "application", "json", MappingJackson2HttpMessageConverter.DEFAULT_CHARSET),
                new MediaType(
                    "text", "javascript", MappingJackson2HttpMessageConverter.DEFAULT_CHARSET)));
      }
    }
    return restTemplate;
  }

  @Bean
  public LocalSessionFactoryBean createLocalSessionFactory(
      @Qualifier("mainDataSource") DataSource dataSource) {
    LocalSessionFactoryBean factory = new LocalSessionFactoryBean();
    factory.setDataSource(dataSource);
    factory.setPackagesToScan(
        new String[] {
          "com.abc.def..entities",
        
          "com.abc.entities"
        });
    factory.setHibernateProperties(hibernateVCommitProperties());
    return factory;
  }

  @Bean
  public LocalSessionFactoryBean createToolSessionFactory(
      @Qualifier("toolsDataSource") DataSource dataSource) {
    LocalSessionFactoryBean factory = new LocalSessionFactoryBean();
    factory.setDataSource(dataSource);
    factory.setPackagesToScan(
        new String[] {
          "com.abc.def..entities",
        
          "com.abc.entities"
        });
    factory.setHibernateProperties(hibernateCommitProperties());
    return factory;
  }

  
  @Scheduled(cron = "${server.cron}")
  public void SyncJobTrigger() throws Exception {
    if (log.isInfoEnabled()) {
     
    }
    JobParameters param =
        new JobParametersBuilder()
            .addString("JobID", String.valueOf(System.currentTimeMillis()))
            .toJobParameters();
    JobExecution execution = jobLauncher.run(populateServerDataJob(), param);
    if (log.isInfoEnabled()) {
      log.info("BATCH tools Server job finished with status: " + execution.getStatus());
    }
  }
  
  //--------------------------------- JOBS ----------------------------------------------
  @Bean
  public Job populateServerDataJob() throws Exception {
    return jobBuilderFactory
        .get("populateData")
        .start(serverRequestData()) // this step's writer is failing
        .next(a)
        .build();
  }

  @Bean
  @StepScope
  public HibernateCursorItemReader<Request> myRequestReader() throws Exception {
    // returns reader and works fine as seen in logs, data is fetched
  }

  @Bean
  @StepScope
  public requestToServerProcessor requestToServerRequestProcessor() {
    return new requestToServerProcessor(mainDataSource);
  }

  // WRITER
  @Bean
  @StepScope
  public HibernateItemWriter<ServerRequestDetails> serverRequestWriter() {
    HibernateItemWriter<ServerRequestDetails> writer = new HibernateItemWriter();
    writer.setSessionFactory(createLocalSessionFactory(mainDataSource).getObject());
    return writer;
  }

  @Bean
  @JobScope
  public Step serverRequestData() throws Exception {
    return stepBuilderFactory
        .get("getServerRequestData")
        .transactionManager(mytransactionManager())/***** added manager here.
        .<Request, ServerRequestDetails>chunk(100)
        .reader(mylRequestReader())
        .processor(requestToServerRequestProcessor())
        .writer(serverRequestWriter())
        .build();
  }
}

DataSourceConfiguration.class
@Configuration
public class DataSourceConfiguration {

  private static final Logger log = LoggerFactory.getLogger(DataSourceConfiguration.class);
  @Value("${spring.datasource.usern}")
  private String insightsDbUser;

  @Value("${spring.datasource.pass}")
  private String insightsDbPass;

  @Value("${spring.datasource.url}")
  private String insightsDbConnection;

 
  @Value("${tools.datasource.user}")
  private String toolsDbUser

  @Value("${tools.datasource.pass}")
  private String toolsDbPass;

  @Value("${tools.datasource.url}")
  private String toolsDbConnection;

  @Bean(name = "mainDataSource")
  @Primary
  public DataSource mainDataSource() {

    DriverManagerDataSource dataSource = new DriverManagerDataSource();

    dataSource.setDriverClassName("com.mysql.jdbc.Driver");
    dataSource.setUrl(insightsDbConnection);
    dataSource.setUsername(insightsDbUser);
    dataSource.setPassword(insightsDbPass);
    if (log.isInfoEnabled()) {
      //
    }
    return dataSource;
  }

  
  @Bean(name = "toolsDataSource")
  public DataSource toolsDataSource() {
    DriverManagerDataSource dataSource = new DriverManagerDataSource();
    dataSource.setDriverClassName("com.mysql.jdbc.Driver");
    dataSource.setUrl(toolsDbConnection);
    dataSource.setUsername(toolsDbUser);
    dataSource.setPassword(toolsDbPass);
    if (log.isInfoEnabled()) {
      //
    }
    return dataSource;
  }
}

Now in this serverRequestData, is the first step of job, and step "a" is second step, step "a" code I have not put because it is exactly similar to step "serverRequestData". I have created a HibernateTransactionManager and specified that while running step, but now first step works fine.Earlier problem with first step was here. Now step "a" fails because of following error which seems similar to earlier failure but I do see extra stackTrace :

2021-04-16 09:56:52.986  INFO 47235 --- [main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [a]
2021-04-16 09:56:59.305  INFO 47235 --- [main] o.s.batch.core.step.tasklet.TaskletStep  : Commit failed while step execution data was already updated. Reverting to old version.
2021-04-16 09:56:59.308 ERROR 47235 --- [main] o.s.batch.core.step.AbstractStep         : Encountered an error executing step getVsyncDetails in job populateVtoolsServerDataJob

javax.persistence.TransactionRequiredException: no transaction is in progress
    at org.hibernate.internal.AbstractSharedSessionContract.checkTransactionNeededForUpdateOperation(AbstractSharedSessionContract.java:413)
    at org.hibernate.internal.SessionImpl.checkTransactionNeededForUpdateOperation(SessionImpl.java:3397)
    at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1354)
    at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1349)
    at org.springframework.orm.hibernate5.SessionFactoryUtils.flush(SessionFactoryUtils.java:148)
    at org.springframework.orm.hibernate5.SpringSessionSynchronization.beforeCommit(SpringSessionSynchronization.java:95)
    at org.springframework.transaction.support.TransactionSynchronizationUtils.triggerBeforeCommit(TransactionSynchronizationUtils.java:96)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerBeforeCommit(AbstractPlatformTransactionManager.java:919)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:727)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
    at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:152)
    at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:273)
    at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:82)
    at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:375)
    at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
    at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:145)
    at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:258)
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:208)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy118.execute(Unknown Source)
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
    at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:410)
    at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:136)
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:319)
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:147)
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140)
    at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:199)
    at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:173)
    at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:160)
    at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:155)
    at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:150)
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:786)
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:776)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:322)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1237)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
    at com.CoreApplication.main(CoreApplication.java:29)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:107)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)

2021-04-16 09:56:59.309  INFO 47235 --- [main] o.s.batch.core.step.AbstractStep         : Step: [a] executed in 6s323ms
2021-04-16 09:56:59.309 ERROR 47235 --- [main] o.s.batch.core.step.AbstractStep         : Encountered an error saving batch meta data for step a in job populateServerDataJob. This job is now in an unknown state and should not be restarted.

org.springframework.dao.OptimisticLockingFailureException: Attempt to update step execution id=2 with wrong version (1), where current version is 2
    at org.springframework.batch.core.repository.dao.MapStepExecutionDao.updateStepExecution(MapStepExecutionDao.java:106)
    at org.springframework.batch.core.repository.support.SimpleJobRepository.update(SimpleJobRepository.java:196)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy93.update(Unknown Source)
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:275)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy118.execute(Unknown Source)
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
    at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:410)
    at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:136)
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:319)
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:147)
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140)
    at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:199)
    at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:173)
    at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:160)
    at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:155)
    at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:150)
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:786)
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:776)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:322)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1237)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
    at com.CoreApplication.main(CoreApplication.java:29)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:107)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)

2021-04-16 09:56:59.313  INFO 47235 --- [main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [SimpleJob: [name=populateServerDataJob]] completed with the following parameters: [{}] and the following status: [UNKNOWN] in 9s178ms

I see one extra log

2021-04-16 09:56:59.305  INFO 47235 --- [main] o.s.batch.core.step.tasklet.TaskletStep  : Commit failed while step execution data was already updated. Reverting to old version.

Which I did not see earlier.

Now since after adding HibernateTransactionManager, the first step writer is working fine, so I added same Transaction Manage while configuring step a. But now I am getting same error but with some extra log, so I think there can be some another reason behind this?

score 0 · Answer 1 · answered Apr 19 '21 at 08:31

0

You are declaring a ResourcelessTransactionManager bean that will be used by default in Spring Batch. Since this transaction manager does not create any transaction, your hibernate item writer fails with this error.

You need to configure Spring Batch to use the HibernateTransactionManager (and remove the ResourcelessTransactionManager). This can be done with a custom Batch configurer as mentioned in the documentation here. In your case, it could be something like:

@Bean
public BatchConfigurer batchConfigurer(DataSource ds, SessionFactory sf) {
    return new DefaultBatchConfigurer(ds) {
        @Override
        public PlatformTransactionManager getTransactionManager() {
            return new HibernateTransactionManager(sf);
        }
    };
}

answered Apr 19 '21 at 08:31

Mahmoud Ben Hassine

28,519
3
32
50

No. I am specifying HibernateTransactionManager when configuring step by using .transactionManager property – user124 Apr 19 '21 at 12:45
Yes but the job repository will be configured with your `ResourcelessTransactionManager`. You seem to be confusing the transaction manager used for the job repository and the one used by steps. You need to make sure that both are using the `HibernateTransactionManager`. The example in my answer is how to achieve that. – Mahmoud Ben Hassine Apr 19 '21 at 12:47
but my job repository is already working fine with ResourcelessTransactionManager. I dont want to touch that component. And for step execution I am explicitly giving the HibernateTransactionManager which is different from the ResourcelessTransactionManager so why would my step execution use ResourcelessTransactionManager.It should use HibernateTransactionManager. I am having two steps and one step is working fine but other is giving the error as I mentioned in question. We can use same transactionManager for different steps i guess? – user124 Apr 19 '21 at 15:41
yes, in this case, you need to set the hibernate transaction manager in each step. – Mahmoud Ben Hassine Apr 19 '21 at 16:12
can u tell me one thing, the transaction manager set during job configuration like I ma doing above, it will be used only by writter? Because of my reader and processor connects to database A and they work fine as I saw in logs, but writter connects to database B and the datasource for my HibernateTransactionManger is set to database B.is it fine?if yes, then why do I see error while running – user124 Apr 19 '21 at 16:36
i tried to remove my hibernate writer from step and added dummy write that just print data on console.still same error.Can you see stack trace once?I think it is different from No transaction in progress? Can it be cause by resourceless transaction manager – user124 Apr 20 '21 at 14:28

Commit failed while step execution data was already updated. Reverting to old version. Spring batch

1 Answers1