1

I use Spring Batch in a project and I'm looking for a way to pre-process all items of a Chunk before the ItemProcessor starts processing. I need to process the items of the chunk together.

There is a ChunkListener, which is called only before the ItemReader starts, but not before the ItemProcessor starts. There is also a ChunkProvider, which fills the chunk with items. But I found no documentation how to grab the filled chunk.

How can I access all items of Chunk after the step finished reading?

sschmeck
  • 7,233
  • 4
  • 40
  • 67
  • I think this defeats the whole point of chunk processing. You might need another step before doing this, rather than doing it like that. Could you elaborate a little on why need the whole chunk ? Like, you need to calculate the average of the chunk or something like that ? – Asoub Mar 10 '17 at 14:52
  • @Asoub The batches read and write to an database. Let’s assume the ItemReader reads 100k records/items of a table and the ItemProcessor needs per item another 100 joined records of other tables. For performance reason I want eager-load the joined records. If I eager-load the related records in the ItemReader, the batch needs to manage 10M records at once, which doesn’t work properly. Therefore I want to eager-load only the 500er item chunks before processing, which results in 0.15M records at once. – sschmeck Mar 10 '17 at 23:15
  • 1
    @sschmek I see. I had a problem like this, but not with a hundred JOINs per record !.. I used a successive temporary tables, with all join already done (maybe ten) and a manual RowMapper (bug-prone, but it was pretty efficient), but I'm not sure this is viable with hundreds of JOIN. What Reader and RowMapper (or ORM) are you currently using ? – Asoub Mar 13 '17 at 16:01
  • Did you find the solution? – hbamithkumara Aug 13 '20 at 11:23

0 Answers0