Spring Batch: Using SkipPolicy and RetryPolicy with SpringBoot

Question

I started learning spring batch recently and doing a POC for a project, Am stuck at retry policy and Skip policy. I have a bit ambiguity in getting solutions for these questions

Question 1): Can we use SkipPolicy and Retry Policy at the same time for an instance, there is an exception at the reader, now framework will look for the policies that we defined. will it be going to both the policies or the one which we defined first..? (I have implemented this in my POC, but it going only to Skip Policy but not to retry policy) What I want to check is, if the exception is an instance of Skippable then do my business logic else if the exception is an instance of Retryable then retry for certain limit.

Question 2): How to get the data for failed records, I know there are listeners(ReadListener, WriteListener, ProcessListener) but, let's say if retry is over the limit, I want to log which record did cause that exception.

Question 3): Is there any way that we can get exact failed records while writing a batch into DB or writing it to someplace(CSV or Flat file)

I did try these scenarios but am unable to get a clear understanding of this.

Any help would be Highly appreciated :)

Here is the Step Configuration

@Bean
public Step step() {
    return stepBuilderFactory.get("eventStep")
            .<Employee, Employee>chunk(3)
            .reader(employeeItemReader())
            .listener(stepItemReadListener)
            .listener(noWorkFoundStepExecutionListener)
            .listener(new StepItemReadListener())
            .processor(processor())
            .writer(writer())
            .faultTolerant()
            .skipPolicy(dbConnectivitySkipper)
            .retryPolicy(stepRetry)
            .listener(stepItemWriteListner)
            .build();
}

Implementation of Skip policy

@Override
public boolean shouldSkip(Throwable exception, int skipCount) throws SkipLimitExceededException {

    if (exception instanceof DuplicateKeyException && skipCount <= 5) {
        return true;
    } else  if (exception instanceof  NumberFormatException && skipCount <= 5){
        return true;
    } else {
        return false;
    }

Implementation of Retry Policy

public boolean canRetry(RetryContext context) {

    Throwable t = context.getLastThrowable();
    if (t instanceof NullPointerException && context.getRetryCount() <= maxAttempts) {
        return true;
    } else  if (t instanceof StepListenerFailedException && context.getRetryCount() <= maxAttempts){
        return true;
    } else {
        return false;
    }
}

Answer 1

There is a lot of details in your question, I will try to provide as much insights as possible:

Question 1:

The retry policy is not applied for the item reader. So even if you declare an exception as retryable and that exception is thrown from the reader, the retry policy is not invoked. That's why you only see the skip policy being applied.

Question 2

If the retry limit is exceeded, then your job will fail. In that case, you can get the last failure exception using JobExecution#getAllFailureExceptions. This last failure exception should give you which item caused the retry limit to be exhausted.

Question 3

When a skippable exception is thrown in the writer, Spring Batch can't know which item caused the chunk to fail. In that case, it will "scan" the chunk by trying items one at a time (it will actually change the chunk size to 1 and issue a transaction for each item). Hence, only the faulty item will be skipped. By using a skip listener, you can get exactly the item you are looking for.

Hope this helps.