0

I am running the spring batch job in three machines. For example the database has 30 records, the batch job in each machine has to pick up unique 10 records and process it.

I read partitioning and Parallel processing and bit confused, which one is suitable?

Appreciate your help.

Rathna
  • 73
  • 1
  • 5

1 Answers1

2

What you are describing is partitioning. Partitioning is when the input is broken up into partitions and each partition is processed in parallel. Spring Batch offers two different ways to execute partitioning, one is local using threads (via the TaskExecutorPartitionHandler). The other one is distributing the partitions via messages so they can be executed either locally or remotely via the MessageChannelPartitionHandler found in Spring Batch Admin's spring-batch-integration project. You can learn more about remote partitioning via my talk on multi-jvm batch processing here: http://www.youtube.com/watch?v=CYTj5YT7CZU

Michael Minella
  • 20,843
  • 4
  • 55
  • 67
  • Thanks Michael. Instead of doing portioning using spring integration can we use Process Indicator patter to achieve parallelism. – Rathna Oct 22 '13 at 22:43
  • I understood that remote chunking or remote portioning is a viable solution for my use case. Thank you very much for your time. – Rathna Oct 23 '13 at 01:48