Retry not working with Spring Batch with Java Config

Question

I have Spring batch job with following config:

@Bean
public Job myJob(Step step1, Step step2, Step step3) {
    return jobs.get("myJob").start(step1).next(step2).next(step3).build();
}

@Bean
public Step step1(ItemReader<String> myReader,
                ItemProcessor<String, String> myProcessor,
                ItemWriter<String> myWriter) {
    return steps.get("step1").<String, String>chunk(1)
            .reader(myReader)
            .faultTolerant().retryLimit(3).retry(MyException.class)
            .processor(myProcessor)
            .writer(myWriter)
            .build();
}

@Bean
@StepScope
public MyReader myReader() {
    return new MyReader();
}
@Bean
public MyProcessor myProcessor() {
    return new MyProcessor();
}
@Bean
public MyWriter myWriter() {
    return new MyWriter();
}

When MyReader class throws MyException it is stopping the execution of the job without retrying with following stack trace:

2019-05-16 14:45:09.460 ERROR 22485 --- [           main] o.s.batch.core.step.AbstractStep         : Encountered an error executing step step1 in job myJob

org.springframework.batch.core.step.skip.NonSkippableReadException: Non-skippable exception during read
    at org.springframework.batch.core.step.item.FaultTolerantChunkProvider.read(FaultTolerantChunkProvider.java:105) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
    at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:116) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]

score 6 · Accepted Answer · answered May 17 '19 at 07:28

6

When MyReader class throws MyException it is stopping the execution of the job without retrying

The retry policy is not applied to the item reader. So even if you declare an exception as retryable and that exception is thrown from the reader, the retry policy is not invoked.

The retry policy is only applied to the processor and writer.

answered May 17 '19 at 07:28

Mahmoud Ben Hassine

28,519
3
32
50

could you please take a look at: https://github.com/atulkulkarni18/spring-batch-reader-retry ? As it seems retry is working even for reader. – Guru May 20 '19 at 11:56
@Mahmoud Ben Hassine - Thanks for the answer, but do you know why the policy is not applied to an item reader? It does not make much sense! – adesai May 20 '19 at 15:02
1

@Atul K In the link you shared, the retry feature is implemented in the reader itself using spring-retry. This is different from the retry policy that you specify in the fault tolerant step which is applied to the [ChunkProcessor](https://docs.spring.io/spring-batch/4.1.x/api/org/springframework/batch/core/step/item/ChunkProcessor.html), which does **not** call the reader. – Mahmoud Ben Hassine May 22 '19 at 07:13
@adesai The reason for that is because the contract of an item reader is "forward only", there is no way (in the current interface) to go back to a previous position in case of a retry. – Mahmoud Ben Hassine May 22 '19 at 07:30
At a high level, a chunk oriented step calls two components: a `ChunkProvider` (provides a chunk by calling the item reader) and a `ChunkProcessor` (processes a chunk by calling the processor + writer). When you specify a retry policy in a fault tolerant step, a `FaultTolerantChunkProcessor` is configured with the retry policy (which is not the case for the chunk provider). Hence, the retry policy is applied to the processor and writer but not for the reader. – Mahmoud Ben Hassine May 22 '19 at 07:36

Guru · Answer 2 · 2019-05-17T08:12:20.183

Seems the retry functionality was pulled out of Spring Batch as of 2.2.0. It is now part of a new library, Spring Retry. https://docs-stage.spring.io/spring-batch/docs/current/reference/html/retry.html#retry

Steps to handle this with declarative retry:

Step 1: Include @EnableRetry in application

Spet 2: Add aop starter based on https://github.com/spring-projects/spring-retry#additional-dependencies

Gradle runtime('org.springframework.boot:spring-boot-starter-aop')

Maven

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
    <scope>runtime</scope>
</dependency>

Spet 3: Include @Retryable in MyReader

@Override
@Retryable(include = { MyException.class }, maxAttempts = 5)
public String read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {

Sample code is checked into github: https://github.com/atulkulkarni18/spring-batch-reader-retry

Sample output:

MyReader : 0
MyProcessor : 0
MyWriter : [0]
MyReader : 1
MyProcessor : 1
MyWriter : [1]
MyReader : 2
MyProcessor : 2
MyWriter : [2]
MyReader : 3
****
MyReader : 3
****
MyReader : 3
****
MyReader : 3
MyProcessor : 3
MyWriter : [3]
MyReader : 4
MyProcessor : 4
MyWriter : [4]
MyReader : 5

@Mahmoud Ben Hassine I have checked in the code in github. Could you please let me know if I am missing something? Seems retry is working in Reader. — Guru, May 17 '19 at 08:14
I want to use faultTolerant for retries as it provides the retry of entire step. and Mahmoud Ben Hassine is right about retries not working with item reader. — adesai, May 20 '19 at 15:04
But @Retryable is working with item reader (based on console output), isn't it? — Guru, May 20 '19 at 15:07

Retry not working with Spring Batch with Java Config

2 Answers2

Linked

Related