Spring Batch custom partitioner for input file

Question

I'm trying to partition a flat input file containing ; separated items. the first item on a line indicates a category and I would like to partition on this category so that for each category a partition is created that will be handled by a dedicated thread. But I'm puzzled to how I can implement this partitioning logic in a custom Partitioner. The partitioning seems to happen before the chunk oriented step, thus before reading, writing, so it looks like I need to read the file in the custom partitioner line by line, get the category field from the line and collect lines with equal categories and create an ExecutionContext for each of these collections?

Am I looking in the right direction? Can someone with experience provide a small example using a file (may be pseudo code)?

Answer 1

I've just hit this question myself. I think that custom Partitioner needs to be paired with a custom ItemReader. ItemReader gets initialized with data from slave step (put there by the Partitioner) and consequently only reads the items that are right for that step.

Answer 2

You can find some custom partitioner implementations in these links here and here to get an overall idea. But I think you can't flee from the ExecutionContext creation for each partition.