Spring Batch serialization problem when using Jackson

Question

While upgrading an application from Spring Boot 1.3.x to 2.1.x I encountered a problem with Spring Batch. Apparently, Spring Batch needs to serialize its execution context and used XStream so far, but switched to Jackson sometime in the version history.

Now, I get an exception when running a job:

2019-10-08 15:50:58:630 ERROR [main] [:] o.s.b.c.j.AbstractJob [] Encountered fatal error executing job
    org.springframework.batch.core.JobExecutionException: Flow execution ended unexpectedly
    at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:142) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:313) [spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:144) [spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:137) [spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    […]
Caused by: org.springframework.batch.core.job.flow.FlowExecutionException: Ended flow=synchronizeBasicCustomerJob at state=synchronizeBasicCustomerJob.writeJobIntoJobContextStep0 with exception
    at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:178) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:136) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    ... 41 more
Caused by: java.lang.IllegalArgumentException: Could not serialize the execution context
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.serializeContext(JdbcExecutionContextDao.java:306) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.updateExecutionContext(JdbcExecutionContextDao.java:146) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.repository.support.SimpleJobRepository.updateExecutionContext(SimpleJobRepository.java:216) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_202]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_202]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_202]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_202]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295) ~[spring-tx-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at com.sun.proxy.$Proxy242.updateExecutionContext(Unknown Source) ~[?:?]
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:159) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:136) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    ... 41 more
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Infinite recursion (StackOverflowError) (through reference chain: org.springframework.batch.core.JobExecution["stepExecutions"]->java.util.Collections$UnmodifiableRandomAccessList[0]->org.springframework.batch.core.StepExecution["jobExecution"]->org.springframework.batch.core.JobExecution["stepExecutions"]->java.util.Collections$UnmodifiableRandomAccessList[0]->org.springframework.batch.core.StepExecution["jobExecution"]->org.springframework.batch.core.JobExecution["stepExecutions"]->java.util.Collections$UnmodifiableRandomAccessList[0]->org.springframework.batch.core.StepExecution["jobExecution"]->org.springframework.batch.core.JobExecution["stepExecutions"]->…

The cause is pretty straightforward: The entity JobExecution has a list of StepExecutions and StepExecution has a backreference to JobExecution, which is cyclic, which causes Jackson to throw this ugly error.

The recommended solution to this problem seems to be to annotate the classes to be serialized with Jackson annotations (@JsonManagedReference and @JsonBackReference). But that is not possible in this context, as the offending classes are Spring Batch's.

Another solution is to configure another ExecutionContextSerializer, i.e. the XStreamExecutionContextStringSerializer. This solution works for me, but the XStreamExecutionContextStringSerializer class is deprecated and I'd rather not use that one.

Other people (Migration to Spring Boot 2 and using Spring Batch 4) have the reverse problem: In their case the deserialization of the persisted execution context fails because it was written by XStream and now can not be read by Jackson. But in my case, the serialization fails, so this is not a compatibility issue between XStream and Jackson.

What am I doing wrong?

You can always add Mixins to actually mix the annotation of an interface/class with the same properties to those classes in the `Spring Batch`. Check https://medium.com/@shankar.ganesh.1234/jackson-mixin-a-simple-guide-to-a-powerful-feature-d984341dc9e2 — Garis M Suero, Oct 08 '19 at 14:19
Possible duplicate of [Migration to Spring Boot 2 and using Spring Batch 4](https://stackoverflow.com/questions/49625231/migration-to-spring-boot-2-and-using-spring-batch-4) — Mahmoud Ben Hassine, Oct 09 '19 at 22:15
@MahmoudBenHassine The post you linked refers to deserialization (Jackson is unable to parse the configuration written by XStream), but my problem is about serialization. — David, Oct 10 '19 at 05:54
ok got it. So you said it works when you use the `XStreamExecutionContextStringSerializer`, but apparently fails with the default jackson config. Correct? The mixin route is a clever work around, but as a Spring Batch user, you are not supposed to do that to make Spring Batch work. Are you storing the jobExecution as an attribute in the execution context? If you can provide a project that reproduces the issue (with an embedded db), I will try to help. — Mahmoud Ben Hassine, Oct 10 '19 at 12:31
The RemotePartitioningWorkerStepBuilderFactory serializes the StepExecution when in aggregating mode when using JSON serialization and it causes this same error. — Chuck C, Mar 09 '22 at 23:30

score 3 · Answer 1 · answered Oct 08 '19 at 14:26

3

As I stated in the comments section, you can use Jackson MixIns to address your issue since you don't have access to the spring batch source code in your project.

It would work like

public abstract class JobExecutionMixin {
  @JsonManagedReference
  private Collection<StepExecution> stepExecutions;
}

And for the back reference:

public abstract class StepExecutionsMixin {
  @JsonBackReference
  private final JobExecution jobExecution;
}

And then, you will have to implement the ObjectMapper bean in your spring boot app with something like:

ObjectMapper mapper = new ObjectMapper();
mapper.addMixInAnnotations(org.springframework.batch.core.StepExecutions.class, StepExecutionsMixin.class);
mapper.addMixInAnnotations(org.springframework.batch.core.JobExecution.class, JobExecutionMixin.class);

answered Oct 08 '19 at 14:26

Garis M Suero

7,974
7
45
68

1

Thank you, I did not know about Jackson Mixins yet. This solution kind of works, but with a (IMO) pretty hefty caveat: To customize the ObjectMapper in the `Jackson2ExecutionContextStringSerializer`, you can simply use the provided setter. But: the default ObjectMapper has the `JobParametersModule` configured which is private. So, in order to add the mixins, I have to effectively copy the `Jackson2ExecutionContextStringSerializer` which i really do not want to do. I wonder how this is supposed to work out of the box in Spring Batch? – David Oct 09 '19 at 06:03
I wouldn't feel too bad about that, spring (probably spring **boot**?) is doing all of that for you automatically. Just find the defaults and implement the bean for the `Jackson2ExecutionContextStringSerializer`. If you didn't have spring boot, you would have to implement that. – Garis M Suero Oct 09 '19 at 18:08
I am in fact using Spring Boot and no such automatism kicks in. I also can not find another implementation of `ExecutionContextSerializer` anywhere. – David Oct 10 '19 at 05:56
I'm facing same issue.. Could not serialize the execution context, have tried adding the below bean in my context but still has the same error. If anyone has any working solution please do comment...Thanks.. – kkk Dec 11 '19 at 12:49

Spring Batch serialization problem when using Jackson

1 Answers1

Linked