SpringBatch: ChunkStep retried on exception in writer even with NeverRetryPolicy

Question

I've configured a ChunkStep in Spring Batch like the following:

stepBuilderFactory
    .get( "myStep" )
    .<I, O> chunk( 1 )
    .transactionManager( txManager )
    .faultTolerant()
    .skipPolicy( new AlwaysSkipItemSkipPolicy() )
    .retryPolicy( new NeverRetryPolicy() )
    .listener( stepSkipListener )
    .reader( someReader ).processor( someProc ).writer( someWriter );

to process one item by the other and in case of an exception in processor or writer to just log it.

Unfortunately the first exception thrown by the writer is not even sent to the listener but the item is retried (processor and writer).

How do I have to configure the chunk step to just skip the item?

Answer 1

Looking at your code, I think you are in a similar situation as in: https://github.com/spring-projects/spring-batch/issues/2071. Quoting Dr Dave Syer from this comment:

A skip is still (and always probably will be) a recovery option for an exhausted retry

On which I agree. So using an "Always skip" policy with a "Never retry" policy is actually incompatible.

Technically speaking, when chunk scanning is activated (ie when a skippable exception is thrown), each item is re-processed/re-written in its own transaction (the commit interval is (re)set to 1 behind the scene and the chunk is reprocessed item by item). In other terms, the ChunkProcessor and more precisely the FaultTolerantChunkProcessor in your case (which is composed of an ItemProcessor + ItemWriter) is re-applied for each item of the chunk.

So you might think your item has been retried (despite using NeverRetryPolicy), but this is actually part of the chunk scanning process (related to the skip policy).

Unfortunately the first exception thrown by the writer is not even sent to the listener

That should not be the case. You can find an example here: https://stackoverflow.com/a/51985616/5019386. Please note that the order of method calls when using stepBuilderFactory might matter as those methods return different types of builders, so I invite you to try:

stepBuilderFactory
    .get( "myStep" )
    .<I, O> chunk( 1 )
    .reader( someReader ).processor( someProc ).writer( someWriter )
    .faultTolerant()
    .skipPolicy( new AlwaysSkipItemSkipPolicy() )
    .listener( stepSkipListener );