0

I have two steps job. Step1 ends up writing to file using AsyncItemWriter, while I am writing to a file I want to check every record if it has any error in it (imagine record object with error flag in it). In step 2, if any of the previously written records has an error in it then I want to copy written file to s3 else delete that written file. Currently I am storing this info in Atomic boolean instance variable in my AsyncItemWriter and the @AfterStep method reading that flag and bubbling it up to the StepExecutionContext and using that in job Flow. I feel like there is a better way for this, any suggestions?

Additional info to the question:

  • I need to process entire file and let my clients know all errors in their file so that they can fix them and re-upload the file
  • I am using AsycnItemProcessor + AsyncItemWriter combination in Step1
Sagar
  • 5,315
  • 6
  • 37
  • 66

1 Answers1

0

I would fail step 1 immediately on the first error, since the file will be discarded anyway (no need to continue reading and processing the rest of the file). Then, with a decider based on the exit status of step 1, I would either delete the file (tasklet) or upload the file to s3 (another tasklet).

Edit: update answer based on comments

I need to process entire file and let my clients know all errors in their file so that they can fix them and re-upload the file

In that case, you can let step 1 continue until the end and use a processor to validate records and add the information in the execution context whenever a record is invalid (validation is btw a typical use case of an item processor, so I believe it is better to use a processor for item validation instead of doing it in the writer). Based on that (ie the execution context), the decider can then decide which step to execute next.

Mahmoud Ben Hassine
  • 28,519
  • 3
  • 32
  • 50
  • Well, my requirement doesn't allow me to fail it immediately. I need to process entire file and let my clients know _all_ errors in their file so that they can fix them and re-upload the file. PS: File will be discarded only if file is successfully processed and doesn't have errors in it. It should be uploaded to S3 only when there are errors in it and as I mentioned, that uploaded file should have _all_ errors in it. – Sagar May 11 '22 at 17:16
  • ok. `I need to process entire file and let my clients know all errors in their file so that they can fix them and re-upload the file` this is an important detail and was not mentioned (or at least not clearly explained) in your description. In that case, you can let step 1 continue until the end and use a processor to validate records and add the info in the execution context. Based on that, the decider can then decide which step to execute next. I will update the answer accordingly. – Mahmoud Ben Hassine May 12 '22 at 10:16
  • I am using AsycnItemProcessor + AsyncItemWriter, what you are suggesting with processor is going to have similar problem since it's AsyncItemProcessor, am I missing something? Sorry If my questions doesn't mention this info, added that to the question. – Sagar May 12 '22 at 14:08