Spring-batch : How to catch exception message with skip method in spring batch?

Question

I am a novice of spring batch. My question is how to catch exceptions with the skip method in spring-batch? As I know, we can use a skip method to skip them when some exceptions happened in spring batch. But how can I get the exception message with skip method? somebody suggested me use SkipListener ,this class has three call back method like onSkipInProcess(),but it is no use for me. And ItemProcessListener did not work either.

The code like below:(I use skip method to ignore the exception and two listeners to receive the exception info)

Step mainStep = stepBuilder.get("run")
        .<ItemProcessing, ItemProcessing>chunk(5)
        .faultTolerant()
        .skip(IOException.class).skip(SocketTimeoutException.class)//skip IOException here
        .skipLimit(2000)
        .reader(reader)
        .processor(processor)
        .writer(writer)
        .listener(stepExecListener)
        .listener(new ItemProcessorListener()) //add process listener
        .listener(new SkipExceptionListener()) //add skip exception listner
        .build();

ItemProcessorListener like below:

//(this class implements ItemProcessListener )
{
    @Override
    public void beforeProcess(Object item) {
        // TODO Auto-generated method stub

    }
    @Override
    public void afterProcess(Object item, Object result) {
        logger.info("invoke remote finished, item={},result={}",item,result);

    }
    @Override
    public void onProcessError(Object item, Exception e) {
        logger.error("invoke remote error, item={},exception={},{}",item,e.getMessage(),e);
    }
}

SkipExceptionListener like below:

//(implements SkipListener<Object, Object>)
{

    @Override
    public void onSkipInRead(Throwable t) {
        // TODO Auto-generated method stub      
    }

    @Override
    public void onSkipInWrite(Object item, Throwable t) {
        // TODO Auto-generated method stub      
    }

    @Override
    public void onSkipInProcess(Object item, Throwable t) {
        logger.info("invoke remote finished,item={},itemJsonStr={},errMsg={},e={}",
                    item,
                    JSONObject.toJSONString(item),
                    t.getMessage(),
                    t);
    }
}

The issue is that all logger did not work. Actually the skip method does work well, I can get the skip count in table batch_step_execution. I am not sure these two listeners whether be callback. Who can tell me how can I do? Or is there anything else? Thanks a lot.

Answer 1

How to catch exception message with skip method in spring batch?

You can do that by implementing the SkipListener interface or extending the SkipListenerSupport class. All methods in the SkipListener interface have a Throwable parameter which is the exception thrown and that caused the item to be skipped. This is where you can get the exception message. Here is an example:

import java.util.Arrays;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.SkipListener;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.support.ListItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableBatchProcessing
public class MyJob {

    @Autowired
    private JobBuilderFactory jobs;

    @Autowired
    private StepBuilderFactory steps;

    @Bean
    public ItemReader<Integer> itemReader() {
        return new ListItemReader<>(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10));
    }

    @Bean
    public ItemWriter<Integer> itemWriter() {
        return items -> {
            for (Integer item : items) {
                System.out.println("item = " + item);
            }
        };
    }

    @Bean
    public ItemProcessor<Integer, Integer> itemProcessor() {
        return item -> {
            if (item.equals(7)) {
                throw new IllegalArgumentException("Sevens are not accepted!!");
            }
            return item;
        };
    }

    @Bean
    public Step step() {
        return steps.get("step")
                .<Integer, Integer>chunk(5)
                .reader(itemReader())
                .processor(itemProcessor())
                .writer(itemWriter())
                .faultTolerant()
                .skip(IllegalArgumentException.class)
                .skipLimit(3)
                .listener(new MySkipListener())
                .build();
    }

    @Bean
    public Job job() {
        return jobs.get("job")
                .start(step())
                .build();
    }

    public static class MySkipListener implements SkipListener<Integer, Integer> {

        @Override
        public void onSkipInRead(Throwable t) {

        }

        @Override
        public void onSkipInWrite(Integer item, Throwable t) {

        }

        @Override
        public void onSkipInProcess(Integer item, Throwable t) {
            System.out.println("Item " + item + " was skipped due to: " + t.getMessage());
        }
    }

    public static void main(String[] args) throws Exception {
        ApplicationContext context = new AnnotationConfigApplicationContext(MyJob.class);
        JobLauncher jobLauncher = context.getBean(JobLauncher.class);
        Job job = context.getBean(Job.class);
        jobLauncher.run(job, new JobParameters());
    }

}

In this example, MySkipListener implements SkipListener and gets the message from the exception as you are trying to do. The example reads numbers from 1 to 10 and skips number 7. You can run the main method and should see the following output:

item = 1
item = 2
item = 3
item = 4
item = 5
item = 6
item = 8
item = 9
item = 10
Item 7 was skipped due to: Sevens are not accepted!!

I hope this helps.

Answer 2

I couldn't get it to work either with implementing SkipListener (would be nice to know why) but in the end I went with the annotation way which is only briefly mentioned in the docs. Also somebody had a similar issue here using the implementation method (question) and the guy in the answer uses this annotation method instead of implementing the interface.

Example bean:

@Component
public class CustomSkippedListener {

    @OnSkipInRead
    public void onSkipInRead(Throwable throwable) {
    }

    @OnSkipInWrite
    public void onSkipInWrite(FooWritingDTO fooWritingDTO, Throwable throwable) {
        LOGGER.info("balabla" + throwable.getMessage());
    }

    @OnSkipInProcess
    public void onSkipInProcess(FooLoaderDTO fooLoaderDTO, Throwable throwable) {
        LOGGER.info("blabla"  + throwable.getMessage());
    }

    private static final Logger LOGGER = LoggerFactory.getLogger(CustomSkippedListener.class);


}

then autowire and include in the step chain as you did.