11

I'm using Spring Batch 3.0.3 configured with annotation to create a batch job that repeats a step an undetermined number of times.

My first step will read into memory a list of items used during the repeating step. I'd like the repeating steps to iterate through this job-scoped list.

How can I configure my job to run the same step x times? I've seen examples in xml of a step specifying the next step to run. I'm thinking I could point two steps to each other in an infinite loop until the list has been iterated. Would this work and is there a way to do it with annotations? Below is my main configuration file with some code that wasn't working.

@ComponentScan(excludeFilters = @Filter(IgnoreDuringScan.class))
@EnableAutoConfiguration
@EnableBatchProcessing
@Loggable
public class BatchCrudConfiguration
{
    @Bean
    public Job batchCRUDJob(JobBuilderFactory jobBuilderFactory, Step[] processSheetSteps)
    {
        JobBuilder jobBuilder = jobBuilderFactory.get("batchCRUDJob").incrementer(new RunIdIncrementer());
        FlowBuilder<FlowJobBuilder> jobFlowBuilder = jobBuilder.flow(processSheetSteps[0]);
        for (int i = 1; i < processSheetSteps.length; i++)
        {
            jobFlowBuilder = jobFlowBuilder.next(processSheetSteps[i]);
        }
        return jobFlowBuilder.end().build();
    }

    @Bean
    public Step[] processSheetSteps(
            StepBuilderFactory stepBuilderFactory,
            RawDataReader[] readers,
            DelegatingWriter writer,
            DelegatingProcessor processor,
            @Value("${batchcrud.chunkSize}") int chunkSize)
    {
        int numberOfReaders = readers.length;
        Step[] steps = new Step[numberOfReaders];
        for (int i = 0; i < numberOfReaders; i++)
        {
            steps[i] = stepBuilderFactory.get("processSheet" + i + "Step").<RawData, DataItem>
                    chunk(chunkSize).reader(readers[i]).processor(processor).writer(writer).build();
        }
        return steps;
    }
Chris Oppedal
  • 373
  • 2
  • 3
  • 11
  • If you include your configuration, we can provide a more concrete answer. – Michael Minella Mar 26 '15 at 20:23
  • 2
    You wouldn't happen to be the Michael Minella who wrote the book on Spring Batch I was reading last night would you? I've added some code. – Chris Oppedal Mar 27 '15 at 15:27
  • 1
    If the book you're reading is Pro Spring Batch...I am. – Michael Minella Mar 27 '15 at 15:52
  • So there still isn't quite enough code there for me to get a full understanding of what you're trying to do. But, from a high level, if you want to repeat a step, then you should be able to write it as a loop. Create step 1 to load your list. Step 2 processes an item in the list. Step 2 has a listener that determines if there are more items to process in the list. If yes, then it returns the exit code that maps to going back to the same step. If not, then it returns the exit code to continue on in the flow (or end the job). – Michael Minella Mar 27 '15 at 15:56
  • Thanks for the response. How would I code step 2 to repeat itself without using xml? Would the listener be of class JobExecutionDecider? Returning an array of steps from my step bean doesn't seem to be working. – Chris Oppedal Mar 27 '15 at 17:31

1 Answers1

14

The way I'd approach this is via the following:

  1. First step loads the list.
  2. Second step processes an item in the list.
  3. The second step has a StepExecutionListener that evaluates if there are more items in the list to process. If so, it returns an ExitStatus that maps to the same step. If not, it returns an ExitStatus that maps to end the job or continue the job (based on the rest of the flow).

For example:

StepExecutionListener 

public class MyListener {

    @Autowired
    private List myItems;

    @AfterStep
    public ExitStatus afterStep(StepExecution stepExecution) {
        if(myItems.size() > 0) {
            return new ExitStatus("CONTINUE");
        }
        else {
            return new ExitStatus("FINISHED");
        }
    }
}

Job Configuration

...
@Bean
public Step step1() {...}

@Bean
public MyListener listener() {..}

@Bean
public Step step2(MyListener listener) {
    return stepBuilder.get("step2")
                .tasklet(myTasklet()) // Replace this piece as needed
                .listener(listener).build();
}

@Bean
public Job job1(Step step1, Step step2) {
    return jobBuilder.get("job1")
                     .start(step1)
                     .next(step2).on("CONTINUE").to(step2).on("FINISHED").end()
                     .build();
}
...

Note I have not tested this code so there may be typos, etc.

Michael Minella
  • 20,843
  • 4
  • 55
  • 67
  • Thank you. This works perfectly if you add another call to end() in the Job bean. – Chris Oppedal Mar 30 '15 at 02:02
  • 1
    So, if i want to continue from `step1`, can i call this `.next(step2).on("CONTINUE").to(step1).on("FINISHED").end()`? – irvana Mar 16 '16 at 07:50
  • 1
    how to do the multiple step2 execution in parallel instead of sequencially in this case, I have list of items in the list which can be executed in parallel – Senthilkumar Annadurai Oct 05 '16 at 10:28
  • 2
    I tried this and get "org.springframework.batch.core.job.SimpleStepHandler - Duplicate step [step2] detected in execution of job=[jobName]. If either step fails, both will be executed again on restart." in the log. Did you also have this issue? Is there a way to fix it? – youreyecheek Oct 12 '17 at 08:41
  • I too am looking for ways to run this in parallel. – jonasespelita Dec 21 '17 at 04:45
  • What should be the equivalent XML configuration in latest spring batch version ? – Deepak Chaudhary Mar 19 '19 at 13:07