0

Usecase:

I have a list of objects holding customer information. I have to make an API call using the customerId from the individual objects in the list.

Approch: Spring Batch Tasklet Implementation

Approch: Spring Batch Tasklet Implementation

Note: List is already available before the job starts.

One Tasklet is defined that takes in the customer ID and makes the call.

I have created a ListPartioner that is diving the list into partitions but I am stuck at - How to pass this data sequentially to the tasklet ?

From the comments

The below allows to pass partitioned data with chunk size to the step of Reader, Writer and Processor. How can I achieve this with a tasklet ?

@Bean(name="asyncStep")
protected Step asyncStep(JobRepository jobRepository, PlatformTransactionManager transactionManager) throws Exception { return new StepBuilder("myjob", jobRepository).
<EmployeeDTO,EmployeeDTO>chunk(2,transactionManager)
.reader(itemReader(null))
 .processor(asyncItemProcessor())
 .writer(asyncItemWriter()) 
.build();
Jazz
  • 11
  • 3
  • I believe this is duplicate of https://stackoverflow.com/questions/73312257/spring-batch-how-to-process-a-list-of-string-in-parallel-and-return-a-subset-of/73387081#73387081. You will find a complete example there. – Mahmoud Ben Hassine Jun 22 '23 at 05:37
  • That helps thank you ! – Jazz Jun 22 '23 at 11:36
  • Also, I ended up asking another question with this - https://stackoverflow.com/questions/76517791/spring-batch-listpartitioner-can-cause-memory-consumption – Jazz Jun 22 '23 at 11:39

1 Answers1

0

I have created a ListPartioner that is diving the list into partitions but I am stuck at - How to pass this data sequentially to the tasklet ?

After you partition the dataset, each worker step can be a chunk-oriented step defined as follows:

  • item reader: iterates over the IDs from the partition
  • item processor: calls the API for the current item
  • item writer: writes the result or what is required to be written
Mahmoud Ben Hassine
  • 28,519
  • 3
  • 32
  • 50
  • Thank you ! I figured out we have to use chunk oriented step here ( reader, processor and writer). This cannot be achieved using a tasklet step as there is no way to pass partitioned data in a chunk of 1 to the tasklet from the partitioner. – Jazz Jun 15 '23 at 18:58
  • That is not accurate, the worker step can be a simple tasklet as well. The chunk size can be set to any value in the worker step, and if set to 1, the it means each partition will be processed one item at a time. – Mahmoud Ben Hassine Jun 16 '23 at 04:26
  • The below allows to pass partitioned data with chunk size to the step of Reader, Writer and Processor. How can I achieve this with a tasklet ? @Bean(name="asyncStep") protected Step asyncStep(JobRepository jobRepository, PlatformTransactionManager transactionManager) throws Exception { return new StepBuilder("myjob", jobRepository) .chunk(2,transactionManager) .reader(itemReader(null)) .processor(asyncItemProcessor()) .writer(asyncItemWriter()) .build(); – Jazz Jun 21 '23 at 05:52
  • Please edit your question and put the code there, it is no readable in comments. – Mahmoud Ben Hassine Jun 21 '23 at 06:04
  • Added in the description. – Jazz Jun 21 '23 at 09:41
  • Please edit your question, not my answer. Add the code in the question:https://stackoverflow.com/help/how-to-ask – Mahmoud Ben Hassine Jun 21 '23 at 12:22
  • Thank you ! Have added my question. Hope it reflects correctly this time. – Jazz Jun 21 '23 at 16:34