Can Spring Retry be used with Spring Batch FlatFileItemReader

Question

I have the following ItemReader:

import org.springframework.batch.item.ExecutionContext;
import org.springframework.batch.item.ItemStreamException;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.LineMapper;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.FileSystemResource;
import org.springframework.retry.annotation.Retryable;
import org.springframework.stereotype.Service;

@Service
public class MyReader extends FlatFileItemReader<Holding> {

    @Autowired
    public MyReader(LineMapper<Holding> lineMapper, File loadFile) {
        setResource(new FileSystemResource(loadFile));
        final int NUMBER_OF_HEADER_LINES = 1;
        setLinesToSkip(NUMBER_OF_HEADER_LINES);
        setLineMapper(lineMapper);
    }

    @Override
    @Retryable(value=ItemStreamException.class, maxAttempts=5,  backoff=@Backoff(delay=1800000))
    public void open(ExecutionContext executionContext) throws ItemStreamException {
                super.open(executionContext);
    }
}

The file to be read (i.e. loadFile) may or may not be available when running the job. If the file is not available, I want the reader to sleep ~30 minutes and then retry opening the file. If after five attempts, the file is not found it can fail as it normally would by throwing ItemStreamException.

Unfortunately, the above code does not attempt to retry opening the file. It throws ItemStreamException on the first call to open and does not retry open.

Can someone please explain how to do this? Note: I do have @EnableRetry on the SpringBootApplication class.

Answer 1

This one works. I made some small changes because I don't know your classes.

build.gradle

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath(
                "org.springframework.boot:spring-boot-gradle-plugin:1.4.0.RELEASE",
        )
    }
}

apply plugin: 'java'
apply plugin: 'spring-boot'

tasks.withType(JavaCompile) {
    sourceCompatibility = JavaVersion.VERSION_1_8
    targetCompatibility = JavaVersion.VERSION_1_8
}

repositories {
    mavenCentral()
}

springBoot {
    mainClass = "test.MyReader"
}

dependencies {
    compile(
            'org.springframework.boot:spring-boot-starter',
            'org.springframework.boot:spring-boot-starter-aop',
            'org.springframework.retry:spring-retry',
    )
}

MyApplication.java

@EnableRetry
@SpringBootApplication
public class MyApplication implements CommandLineRunner {
    private final MyReader myReader;

    @Autowired
    public MyApplication(MyReader myReader) {
        this.myReader = myReader;
    }

    public static void main(String[] args) {
        SpringApplication.run(MyApplication.class, args);
    }

    @Override
    public void run(String... args) throws Exception {
        myReader.read();
    }
}

MyReader.java

@Service
public class MyReader {
    private static final String PATH = "a2";
    private final Logger logger = LoggerFactory.getLogger(MyReader.class);

    public MyReader() {

    }

    @Retryable(value = IOException.class, maxAttempts = 5, backoff = @Backoff(delay = 5000))
    public void read() throws IOException {
        final Resource resource = new FileSystemResource(PATH);
        logger.info("\n\nRead attempt: {}\n", resource);
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(resource.getInputStream(), StandardCharsets.UTF_8))) {
            final String content = reader.lines().collect(Collectors.joining(" "));
            logger.info("\n\nFile content: {}\n", content);
        }
    }
}

When you execute and file is there, you see in logs one "Read attempt" message and one "File content" message. I even added empty lines in there so it's hard to overlook now.

When file does not exists you will see five "Read attempt" messages and then exception thrown.

I changed retry time to 5 seconds. If you are fast enough you can start without file and then make some file in there and you will see it works. You should see couple of read attempts and finally the file content.

I can see you are struggling for couple of days with this issue already. Please do not spawn unnecessary questions, as they are not helping the community. For the future reference, try to stick to your one question, modifying it if need arises.

Answer 2

Moving from Spring Boot version 1.3.1.RELEASE to 1.4.0.RELEASE (and its corresponding auto versioned dependencies such as spring-boot-starter-batch) resolved the issue. Retry works in 1.4.0.RELEASE as implemented in the OP. Does not work in 1.3.1.RELEASE. Here's the gradle file that is now being used:

buildscript {
    ext {
        // Previously using 1.3.1.RELEASE where retry functionality does not work
        springBootVersion = '1.4.0.RELEASE' 
    }
    repositories {
        mavenCentral()
        mavenLocal()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
    }
}

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'spring-boot'

configurations {
   provided
}

sourceSets {
    main {
        compileClasspath += configurations.provided
    }
}

jar {
    baseName = 'load'
    version = '1.0'
}
sourceCompatibility = 1.8
targetCompatibility = 1.8

repositories {
    mavenCentral()
    mavenLocal()
}

dependencies {
    compile('org.springframework.boot:spring-boot-starter-batch')
    compile('org.springframework.boot:spring-boot-configuration-processor')
    compile('org.springframework.boot:spring-boot-starter-data-jpa')
    compile('org.springframework.boot:spring-boot-starter-mail')
    compile('org.springframework.boot:spring-boot-starter-aop')
    compile('org.projectlombok:lombok:1.16.6')
    compile('org.hibernate:hibernate-validator:5.2.4.Final')
    compile('org.quartz-scheduler:quartz:2.2.3')
    runtime('javax.el:javax.el-api:2.2.4')
    runtime('org.glassfish.web:javax.el:2.2.4')
    runtime('net.sourceforge.jtds:jtds:1.3.1')
    testCompile('org.springframework.boot:spring-boot-starter-test')
    testCompile('org.springframework.batch:spring-batch-test')
}



eclipse {
    classpath {
         containers.remove('org.eclipse.jdt.launching.JRE_CONTAINER')
         containers 'org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.8'
    }
}

task wrapper(type: Wrapper) {
    gradleVersion = '2.11'
} 

Note: Considered using JobExecutionDecider to retry step using the FlatFileItemReader. However, the ItemStreamException causes the entire job and application to terminate without the decider getting a chance to execute.