2

I followed the tutorial https://spring.io/guides/gs/batch-processing/ which basically reads a spreadsheet and insert the data in database once the application is started. I want to execute the reading process every time a rest service is evoked, so I added a controller in this application. To call the job I followed the suggestion in How to trigger a job using a rest web service? and now it rises my issue: whenever I called the rest service it runs only the afterJob method. I read how to select which spring batch job to run based on application argument - spring boot java config (which has certain similiarity with my issue) and the excellent blog post pointed in it but I am still stuck. I want to call "public ItemReader reader()" after the rest service and I expect the same flow to be followed as it is done when the application started via main(). I mean, the same flow as when it is deployed by springboot. I guess my confusion lies in @EnableBatchProcessing or JobExecutionListenerSupport but I am really stuck on it. Below are the most important snippet.

Controller

@Autowired
JobLauncher jobLauncher;

@Autowired
Job job;

@RequestMapping("/runit")
public void handle() throws Exception{
       JobParameters jobParameters =
                       new JobParametersBuilder()
                       .addLong("time",System.currentTimeMillis()).toJobParameters();
        jobLauncher.run(job, jobParameters);
}

Listener

@Component
public class JobCompletionNotificationListener extends   
    JobExecutionListenerSupport {

    @Override
    public void afterJob(JobExecution jobExecution) {
        if(jobExecution.getStatus() == BatchStatus.COMPLETED) {

Batch Configuration

@Configuration
@EnableBatchProcessing
public class BatchConfiguration {
    // tag::readerwriterprocessor[]
    @Bean
    public ItemReader<Person> reader() {
        FlatFileItemReader<Person> reader = new FlatFileItemReader<Person>();
        reader.setResource(new ClassPathResource("sample-data.csv"));
        reader.setLineMapper(new DefaultLineMapper<Person>() {{
            setLineTokenizer(new DelimitedLineTokenizer() {{
                setNames(new String[] { "firstName", "lastName" });
            }});

            setFieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {{
                setTargetType(Person.class);
            }});
        }});

        return reader;
    }

    @Bean
    public ItemProcessor<Person, Person> processor() {
        return new PersonItemProcessor();
    }

    @Bean
    public ItemWriter<Person> writer(DataSource dataSource) {
        JdbcBatchItemWriter<Person> writer = new JdbcBatchItemWriter<Person>();
        writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<Person>());

        writer.setSql("INSERT INTO people (first_name, last_name) VALUES (:firstName, :lastName)");

        writer.setDataSource(dataSource);
        return writer;
    }

    @Bean
    public Job importUserJob(JobBuilderFactory jobs, Step s1, JobExecutionListener listener) {
        return jobs.get("importUserJob")
                .incrementer(new RunIdIncrementer())
                .listener(listener)
                .flow(s1)
                .end()
                .build();
    }

    @Bean
    public Step step1(StepBuilderFactory stepBuilderFactory, ItemReader<Person> reader,
            ItemWriter<Person> writer, ItemProcessor<Person, Person> processor) {
        return stepBuilderFactory.get("step1")
                .<Person, Person> chunk(10)
                .reader(reader)
                .processor(processor)
                .writer(writer)
                .build();
    }

application.properties

application.properties
# SPRING BATCH (BatchDatabaseInitializer)
spring.batch.job.enabled=false
spring.batch.initializer.enabled=false
Community
  • 1
  • 1
Jim C
  • 3,957
  • 25
  • 85
  • 162
  • Is there a reason you aren't using Spring Batch Admin to launch jobs via REST? – Michael Minella Jul 09 '15 at 23:06
  • If I understood correctly, you are suggesting me to remove Spring Boot and use Spring Batch Admin instead. Honestly, this is my first time using Spring Batch so, if you can point the advantage to a diferent approach from what I am trying or if you can point what you see strange in my approach I will be thankfull. Although you see a simple example above, I am preparing to replace important jobs next month done by cobol, so, I have to be very assertive. All batchs will run in Mainframe, so I found very interesting the purpose of Spring boot to rise an embeded tomcat plus Spring Batch features. – Jim C Jul 10 '15 at 02:06
  • 2
    Not exactly. You can use Spring Boot and Spring Batch Admin together. Take a look at https://github.com/spring-projects/spring-batch-admin-samples – Michael Minella Jul 10 '15 at 02:08
  • I saw this example but it is not clear how it would fix my issue. I can't expose Spring Batch Admin to final user in my company. The batchs belongs to Application A domain and they will start after an user does certain workflow in Application B, e.g. Application B will evoke a rest web service from Application A sending a pojo filled in, then, the rest service must evoke the batch. – Jim C Jul 10 '15 at 02:21
  • The java piece of the angular sample would represent Application A. It contains the jobs and the REST service. Application B would call it sending the launch request. – Michael Minella Jul 10 '15 at 03:52
  • I am studing the spring-batch-admin-angularjs but maybe because I know almost nothing about AngularJS it seems to be a long way for me. Supposing I can't use spring-batch-admin. What do you suggest me? – Jim C Jul 10 '15 at 04:55
  • So what do you mean by "whenever I called the rest service it runs only the afterJob method"? Are you saying that the job itself doesn't run or? The configuration methods shouldn't run again (they are only run once). – Michael Minella Jul 10 '15 at 14:45
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/82979/discussion-between-jim-c-and-michael-minella). – Jim C Jul 10 '15 at 16:35

1 Answers1

9

After a chat that got deeper into this issue, the real "issue" is that the ItemReader is being created once instead of once per REST call (aka once per execution of the job). To fix this, make the ItemReader step scoped. This will allow you to get a new instance of the reader with each run which also enables the ability to inject the file name base on job parameters.

From a code perspective, change this:

@Bean
public ItemReader<Person> reader() {

to this:

@Bean
@StepScope
public FlatFileItemReader<Person> reader() {

The reason for the return type change is that Spring Batch will automatically register ItemStreams for you. However when using @StepScope we only see the return you define. In this case, ItemReader doesn't extend/implement ItemStream so we don't know you're returning something we should be auto-registering. By returning FlatFileItemReader we can see all the interfaces/etc and can apply the "magic" for you.

Michael Minella
  • 20,843
  • 4
  • 55
  • 67
  • A last question, should I worry about "get a new instance of the reader with each run"? I mean, is there any pitfall when I publish this in production? I mean, should I take care of "killing" each instance? – Jim C Jul 10 '15 at 21:12
  • 1
    Nope. Each instance should be garbage collected at the appropriate time. – Michael Minella Jul 10 '15 at 21:14