8

I have followed the spring batch doc and couldn't get my job running Asynchronously.

So I am running the Job from a web container and the job will be triggered via a REST end point.

I wanted to get the JobInstance ID to pass it in response before completing the whole job. So they can check the status of the job later with the JobInstance ID instead of waiting. But I couldn't get it work. Below is the sample code I tried. Please let me know what am I missing or wrong.

BatchConfig to make Async JobLauncher

@Configuration
public class BatchConfig {

    @Autowired
    JobRepository jobRepository;


    @Bean
    public JobLauncher simpleJobLauncher() throws Exception {
        SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
        jobLauncher.setJobRepository(jobRepository);
        jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
        jobLauncher.afterPropertiesSet();
        return jobLauncher;
    }
}

Controller

@Autowired
JobLauncher jobLauncher;

@RequestMapping(value="/trigger-job", method = RequestMethod.GET)
public Long workHard() throws Exception {
    JobParameters jobParameters = new JobParametersBuilder().
            addLong("time", System.currentTimeMillis())
            .toJobParameters();
    JobExecution jobExecution = jobLauncher.run(batchComponent.customJob("paramhere"), jobParameters);
    System.out.println(jobExecution.getJobInstance().getInstanceId());
    System.out.println("OK RESPONSE");
    return jobExecution.getJobInstance().getInstanceId();
}

And JobBuilder as component

@Component
public class BatchComponent {

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    public Job customJob(String someParam) throws Exception {
        return jobBuilderFactory.get("personProcessor")
                .incrementer(new RunIdIncrementer()).listener(listener())
                .flow(personPorcessStep(someParam)).end().build();
    }


    private Step personPorcessStep(String someParam) throws Exception {
        return stepBuilderFactory.get("personProcessStep").<PersonInput, PersonOutput>chunk(1)
                .reader(new PersonReader(someParam)).faultTolerant().
                        skipPolicy(new DataDuplicateSkipper()).processor(new PersonProcessor())
                .writer(new PersonWriter()).build();
    }


    private JobExecutionListener listener() {
        return new PersonJobCompletionListener();
    }

    private class PersonInput {
        String firstName;

        public PersonInput(String firstName) {
            this.firstName = firstName;
        }

        public String getFirstName() {
            return firstName;
        }

        public void setFirstName(String firstName) {
            this.firstName = firstName;
        }
    }

    private class PersonOutput {
        String firstName;

        public String getFirstName() {
            return firstName;
        }

        public void setFirstName(String firstName) {
            this.firstName = firstName;
        }
    }

    public class PersonReader implements ItemReader<PersonInput> {
        private List<PersonInput> items;
        private int count = 0;

        public PersonReader(String someParam) throws InterruptedException {
            Thread.sleep(10000L); //to simulate processing
            //manipulate and provide data in the read method
            //just for testing i have given some dummy example
            items = new ArrayList<PersonInput>();
            PersonInput pi = new PersonInput("john");
            items.add(pi);
        }

        @Override
        public PersonInput read() {
            if (count < items.size()) {
                return items.get(count++);
            }
            return null;
        }
    }


    public class DataDuplicateSkipper implements SkipPolicy {

        @Override
        public boolean shouldSkip(Throwable exception, int skipCount) throws SkipLimitExceededException {
            if (exception instanceof DataIntegrityViolationException) {
                return true;
            }
            return true;
        }
    }


    private class PersonProcessor implements ItemProcessor<PersonInput, PersonOutput> {

        @Override
        public PersonOutput process(PersonInput item) throws Exception {
            return null;
        }
    }

    private class PersonWriter implements org.springframework.batch.item.ItemWriter<PersonOutput> {
        @Override
        public void write(List<? extends PersonOutput> results) throws Exception {
            return;
        }
    }

    private class PersonJobCompletionListener implements JobExecutionListener {
        public PersonJobCompletionListener() {
        }

        @Override
        public void beforeJob(JobExecution jobExecution) {

        }

        @Override
        public void afterJob(JobExecution jobExecution) {
            System.out.println("JOB COMPLETED");
        }
    }
}

Main Function

@SpringBootApplication
@EnableBatchProcessing
@EnableScheduling
@EnableAsync
public class SpringBatchTestApplication {

    public static void main(String[] args) {
        SpringApplication.run(SpringBatchTestApplication.class, args);
    }
}

I am using annotation based configurations and use gradle with the below batch package.

compile('org.springframework.boot:spring-boot-starter-batch')

Please let me know if some more info needed. I couldn't find any example to run this common use case.

Thanks for you time.

rram
  • 2,004
  • 5
  • 24
  • 37
  • You get any exception ? – Abder KRIMA Dec 09 '18 at 05:30
  • No I don't get any Exception @TinyOS – rram Dec 09 '18 at 05:34
  • your configuration seems correct to me. Since you configured a asynchronous task executor in your jobLauncher, it should return immediately the job execution and run the job in a separate thread. Can you elaborate on `couldn't get my job running Asynchronously`? – Mahmoud Ben Hassine Dec 09 '18 at 22:53
  • I am `not` getting Job Instance ID immediately in the controller response, so that it can be used later to check the status of the job. But here the statement is ran only after the job completed fully. @MahmoudBenHassine – rram Dec 10 '18 at 00:13
  • In this case, make sure the correct `JobLauncher` (the one configured with a asynchronous task executor) is injected in your controller. Probably you have another jobLauncher with a synchronous task executor that runs the job until completion before returning the job execution. – Mahmoud Ben Hassine Dec 10 '18 at 12:39
  • @MahmoudBenHassine thanks. After your suggestion I used the bean with `Qualifier` but still the same, I get ID only when the job completes. Here is what I get in IDE console when I hit the controller – rram Dec 10 '18 at 14:36
  • `2 OK RESPONSE 2018-12-10 20:04:58.726 INFO 82585 --- [cTaskExecutor-2] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=personProcessor]] launched with the following parameters: [{time=1544452488708}] 2018-12-10 20:04:58.740 INFO 82585 --- [cTaskExecutor-2] o.s.batch.core.job.SimpleStepHandler : Executing step: [personProcessStep] JOB COMPLETED ` – rram Dec 10 '18 at 14:39
  • `2018-12-10 20:04:58.754 INFO 82585 --- [cTaskExecutor-2] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=personProcessor]] completed with the following parameters: [{time=1544452488708}] and the following status: [COMPLETED]` – rram Dec 10 '18 at 14:39

7 Answers7

15

Try this,In your Configuration You need to create customJobLauncher with SimpleAsyncTaskExecutor using the @Bean(name = "myJobLauncher") and same will be used @Qualifier in your controller.

@Bean(name = "myJobLauncher")
public JobLauncher simpleJobLauncher() throws Exception {
    SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
    jobLauncher.setJobRepository(jobRepository);
    jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
    jobLauncher.afterPropertiesSet();
    return jobLauncher;
}

In your Controller

@Autowired
@Qualifier("myJobLauncher")
private JobLauncher jobLauncher;
  • Please don't write code-only answers, try to elaborate by explaining what you're doing. – JJJ Mar 23 '19 at 11:22
6

If I look at your code I see a couple of mistake. First of all your custom config is not loaded, because, if it was, the injection will fail for duplicate bean instance for the same interface.

There's a lot of magic in spring boot, but if you don't tell him to do some component scan, nothing will be loaded as espected.

The second problem that i can see is your BatchConfig class: it does not extends DefaultBatchConfigure, nor overrides getJobLauncher(), so even if the boot magic will load everything you'll get the default one. Here is a configuration that will work and it's compliant with the documentation @EnableBatchProcessing API

BatchConfig

@Configuration
@EnableBatchProcessing(modular = true)
@Slf4j
public class BatchConfig extends DefaultBatchConfigurer {

  @Override
  @Bean
  public JobLauncher getJobLauncher() {
    try {
      SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
      jobLauncher.setJobRepository(getJobRepository());
      jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
      jobLauncher.afterPropertiesSet();
      return jobLauncher;

    } catch (Exception e) {
      log.error("Can't load SimpleJobLauncher with SimpleAsyncTaskExecutor: {} fallback on default", e);
      return super.getJobLauncher();
    }
  }
}

Main Function

@SpringBootApplication
@EnableScheduling
@EnableAsync
@ComponentScan(basePackageClasses = {BatchConfig.class})
public class SpringBatchTestApplication {

    public static void main(String[] args) {
        SpringApplication.run(SpringBatchTestApplication.class, args);
    }
}
ElPysCampeador
  • 399
  • 1
  • 4
  • 14
  • @Slf4j it's a Lombok annotation, you can ignore it anche inject Logger log manually – ElPysCampeador Sep 19 '19 at 16:45
  • This creates a circular dependency between the BatchConfig file and ModularBatchConfiguration from spring batch. To make it work just remove modular=true from the annotation in BatchConfig – Luca Pasini Jun 20 '21 at 15:06
2

Although you’ve your custom jobLauncher, you’re running the job using default jobLauncher provided by Spring. Could you please autowire simpleJobLauncher in your controller and give it a try?

roschach
  • 8,390
  • 14
  • 74
  • 124
Reddy_73
  • 134
  • 11
  • 1
    I agree, his only problem is he was autowiring the default jobLauncher instead of the simpleJobLauncher by name he created a bean for. – LethalLima Dec 12 '19 at 04:03
2

I know that this is an old question but I post this answer anyway for future users.

After reviewing your code I can't tell why you have this problem, but I can suggest you to use a Qualifier annotation plus use the ThreadPoolTaskExecutor like so and see if it solve your problem.

You may also check this tutorial: Asynchronous Spring Batch Job Processing for more details. It will help you configure a spring batch job asynchronously. This tutorial was written by me.

@Configuration
public class BatchConfig {
 
 @Autowired
 private JobRepository jobRepository;
 
 @Bean
 public TaskExecutor threadPoolTaskExecutor(){
  
  ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setMaxPoolSize(12);
        executor.setCorePoolSize(8);
        executor.setQueueCapacity(15);
  
   return executor;
 }
 
 @Bean
    public JobLauncher asyncJobLauncher() throws Exception {
        SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
        
        jobLauncher.setJobRepository(jobRepository);
        jobLauncher.setTaskExecutor(threadPoolTaskExecutor());
        return jobLauncher;
 }
}
Samuel Liew
  • 76,741
  • 107
  • 159
  • 260
Mohamed Makkaoui
  • 393
  • 6
  • 10
1

JobExecution jobExecution = jobLauncher.run(batchComponent.customJob("paramhere"), jobParameters);. Joblauncher will wait after the Job has been completed before returning anything, that why your service is probably taking long to respond if that is your problem. If you want asynchronous capabilities, you might want to look at Spring's @EnableAsync & @Async.

@EnableAsync

Karl Alexander
  • 351
  • 4
  • 16
  • @EnableAsync is put on application level already in my main function. I need to return job Id in response as well. How to do that? – rram Dec 09 '18 at 09:54
  • Alright, my apologies i didnt catch that. You seem like you already have the right setup, except for annotating your `workHard()` method with @Async as well. `SimpleAsyncTaskExecutor()` should also instantly return the JobExecution with `ExitStatus=Unknown` as stated [here](https://docs.spring.io/spring-batch/4.0.x/reference/html/job.html#configuringJobLauncher) – Karl Alexander Dec 09 '18 at 10:42
  • If I put `@Async` in the controller method `workHard()`. I am not at all getting Job Instance Id but the request is finished instantly with empty response. – rram Dec 09 '18 at 11:25
1

According to spring documentation to return a response of the http request asynchronous it is required to use org.springframework.core.task.SimpleAsyncTaskExecutor.

Any implementation of the spring TaskExecutor interface can be used to control how jobs are asynchronously executed.

spring batch documentation

<bean id="jobLauncher"
  class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<property name="taskExecutor">
    <bean class="org.springframework.core.task.SimpleAsyncTaskExecutor" />
</property>

jkamcc
  • 361
  • 2
  • 3
0

If you're using Lombok this might help you:

TLDR: Lombok @AllArgsConstructor doesn't seem to work well with @Qualifier annotation EDIT: if you have enable @Qualifier annotations in the lombok.config file to be able to use @Qualifier with @AllArgsConstructor like this:

lombok.copyableAnnotations += org.springframework.beans.factory.annotation.Qualifier

I know old question, however I had the exact same problem and none of the answers solved it.

I configured the async job launcher like this and added the qualifier to make sure this jobLauncher is injected:

 @Bean(name = "asyncJobLauncher")
 public JobLauncher simpleJobLauncher(JobRepository jobRepository) throws Exception {
        SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
        jobLauncher.setJobRepository(jobRepository);
        jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
        jobLauncher.afterPropertiesSet();
        return jobLauncher;
    }

And injected it like this

@Qualifier("asyncJobLauncher")
private final JobLauncher jobLauncher;

I was using Lombok @AllArgsConstructor after changing it to autowire, the correct job launcher got injected and the job is now executed asynchronously:

@Autowired
@Qualifier("asyncJobLauncher")
private JobLauncher jobLauncher;

Also I didn't had to extend my configuration from DefaultBatchConfigurer

Sepultura
  • 997
  • 1
  • 9
  • 28