0

I am new to Spring batch with spring boot and read almost all the possible available article but not found exact scenarios/snippets for use case as below.

  1. Read data from DB (for example : 5 Items)
  2. Process the data : Call API and if API call Fails skip the Item and process the next item. (For all 5 items in sequence)
  3. All skipped items should be logged in db
  4. After Processing all the 5 items with failed/success scenario then retry failed items after some time interval.

I have used the RetryTemplate but there it retries the item on exception immediately so it's not working for me.

I have created skip policy which skips the exception and gave me the expected behavior but not able start the retry attempts(3) the same way..

Can somebody please help with some examples and guide me on the same...

Sabir Khan
  • 9,826
  • 7
  • 45
  • 98
Ganesh
  • 903
  • 1
  • 8
  • 18
  • for **All skipped items should be logged in db** , would this happen to same table from where data were read ( by updating some flag ) or to some other table ? – Sabir Khan Oct 29 '18 at 11:05
  • @SabirKhan : yeah to the same table – Ganesh Oct 29 '18 at 11:13
  • so items skipped once would remain marked as **skipped** for forever or you wish to unmark it as **processed** after successful processing ? – Sabir Khan Oct 29 '18 at 11:14
  • @SabirKhan : Actually i am working on bulk data so I want to maintain the skip items in DB with skip flag and after some time interval i want retry those skip items 3 times and after that i will marked them as Failed if at all processing will not work as expected . – Ganesh Oct 29 '18 at 12:23

1 Answers1

0

After Processing all the 5 items with failed/success scenario then retry failed items after some time interval

Fault tolerance features (skip and retry) are executed on items within a chunk, not on the whole chunk. So it is not possible to process all items (where some of them might fail), then once the whole chunk is processed, retry the failed ones. When you specify a retry policy, this policy will be applied on items, not the whole chunk.

I have used the RetryTemplate but there it retries the item on exception immediately

You can set a BackoffPolicy on the retry template and there will be a back off period between each retry attempt (instead of immediate retry). But again, this policy will be applied for the item being retried, not for the whole chunk.

One technique you can use (since the input is a database table) is a variant of the "process indicator pattern" (shown in this sample). The idea is to use a flag to indicate records to be processed. Your job should read only records to be processed, mark successful records as processed (update the flag) and skipped ones as unprocessed (or simply don't update the flag since the default is false). This way, skipped records will be part of the input of the next run of the job and will be retried by design.

Hope this helps.

Mahmoud Ben Hassine
  • 28,519
  • 3
  • 32
  • 50