0

I'm developing a migration app to migrate data from one database to another database, with different schema's - so we have some processors in between. Currently, we're using the JdbcCursorItemReader in our steps. I'm trying to avoid a temporary network issue resulting in a failed job, hours into a migration job.

I tried extending the JdbcCursorItemReader and overriding the 'open(ExecutionContext ec)' and also the 'read()' methods and annotating them with @Retryable. However, when an exception is thrown in either the open or read methods, the job fails - the exception is not caught and retry is not triggered.

I'm now starting to wonder if the JdbcCursorItemReader can encounter transient errors, which would need to be retried. As I understand it, a long running connection is opened and the results are streamed. Am I wasting my time trying to make the JdbcCursorItemReader retryable? Is it even possible?

If I used the JdbcPagingItemReader, could I make its read() method retryable?

I'm not too experienced with Spring Batch, any guidance on making the reader more resilient would be greatly appreciated!

Regards, David

user3008280
  • 3
  • 1

1 Answers1

0

The retry policy of a fault-tolerant chunk-oriented step is not applied to the item reader, see Retry not working with Spring Batch with Java Config and Spring Batch: Using SkipPolicy and RetryPolicy with SpringBoot. The reason for that is because the contract of an item reader is "forward only", there is no way (in the current interface) to go back to a previous position in case of a retry.

So the path you are exploring is the way to go. You can add retry capabilities to your custom reader either declaratively with an annotation or programmatically with a RetryTemplate.

Mahmoud Ben Hassine
  • 28,519
  • 3
  • 32
  • 50
  • Hi Mahmoud, thanks for your feedback. The JdbcCursorItemReader probably doesn't need to been made retryable, as it keeps a connection open and would avoid transient errors that would normally be retried? But if we were to switch to another reader, such as the JdbcPagingItemReader, we could potentially wrap it in a custom reader and make it retryable? – user3008280 Feb 03 '23 at 10:33
  • Good point. Long running cursors are not suitable for all cases, and this is one of them. The paging reader does not hold a long running cursor, it opens a connection, gets a page or data and closes the connection. And indeed, this kind of readers is more suitable to be made retryable. – Mahmoud Ben Hassine Feb 03 '23 at 14:25