0

Is there any way to have the results returned by a Reader in one step passed to later steps?

We have a Job in whih Step 1 retrieves transaction status updates from a legacy stored proc. Some of the results (all but the last update for each transaction) need to be discarded, so the Step 1 ItemProcessor builds a Map with the keys of those "last updates" that need to be processed (Step 1 has a dummy writer).

The Step 2 Reader then calls the same stored proc, and its ItemProcessor discards any results whose keys are not found in the Map. The Step 2 Writer then produces the output file.

Modifying the proc isn't an option, and holding all the results in memory would exceed the 2.5G or so that JVMs can handle on a Windows-based server.

Eliminating that second retrieval would be ideal. Is there a way to do that?

JeffInTampa
  • 55
  • 1
  • 10
  • if modify source table(s) is an option check http://stackoverflow.com/questions/31909584/does-spring-batch-have-a-feature-to-keep-track-of-processed-rows – Luca Basso Ricci Aug 21 '15 at 12:16
  • Thanks, Luca. That's not exactly what I need, however. The results are from multiple tables so a new table would need to to be defined to hold them across jobs, which the DBAs have already nixed. – JeffInTampa Aug 21 '15 at 23:04
  • what about using a small in-memory database (http://stackoverflow.com/questions/3444265/simple-and-reliable-in-memory-database-for-fast-java-integration-tests-with-supp) to hold key-value pair instead of memory? – Luca Basso Ricci Aug 22 '15 at 13:11

0 Answers0