2

I am trying to implement a file reading layer using Spring batch annotation based configuration. I want a new ItemWriter instance to be used for multiple executions of the same job. I assumed that using JobScope would make sure a new writer is used per execution of the same job even if two executions are started at the same time.

My configuration class is as follows :

@Configuration
@EnableBatchProcessing
public abstract class AbstractFileLoader<T> {

    private static final String FILE_PATTERN = "*.dat";


    @Bean
    @StepScope
    @Value("#{stepExecutionContext['fileName']}")
    public FlatFileItemReader<T> reader(String file) {
        FlatFileItemReader<T> reader = new FlatFileItemReader<T>();
        String path = file.substring(file.indexOf(":") + 1, file.length());
        FileSystemResource resource = new FileSystemResource(path);
        reader.setResource(resource);
        DefaultLineMapper<T> lineMapper = new DefaultLineMapper<T>();
        lineMapper.setFieldSetMapper(getFieldSetMapper());
        DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer(",");
        tokenizer.setNames(getColumnNames());
        lineMapper.setLineTokenizer(tokenizer);
        reader.setLineMapper(lineMapper);
        reader.setLinesToSkip(1);
        return reader;
    }

    @Bean
    public ItemProcessor<T, T> processor() {
        // TODO add transformations here
        return null;
    }

    //Exception when using JobScope for the writer
    @Bean
    @JobScope
    public ItemWriter<T> writer() {
        ListItemWriter<T> writer = new ListItemWriter<T>();
        return writer;
    }


    @Bean
    public Job loaderJob(JobBuilderFactory jobs, Step s1,
            JobExecutionListener listener) {
        return jobs.get(getLoaderName()).incrementer(new RunIdIncrementer())
                .listener(listener).start(s1).build();
    }

    @Bean
    public Step readStep(StepBuilderFactory stepBuilderFactory,
            ItemReader<T> reader, ItemWriter<T> writer,
            ItemProcessor<T, T> processor, TaskExecutor taskExecutor,
            ResourcePatternResolver resolver) {

        final Step readerStep = stepBuilderFactory
                .get(getLoaderName() + " ReadStep:slave").<T, T> chunk(25254)
                .reader(reader).processor(processor).writer(writer)
                .taskExecutor(taskExecutor).throttleLimit(16).build();

        final Step partitionedStep = stepBuilderFactory
                .get(getLoaderName() + " ReadStep:master")
                .partitioner(readerStep)
                .partitioner(getLoaderName() + " ReadStep:slave",
                        partitioner(resolver)).taskExecutor(taskExecutor)
                .build();

        return partitionedStep;

    }


    @Bean
    public TaskExecutor taskExecutor() {
        return new SimpleAsyncTaskExecutor();
    }

    @Bean
    public Partitioner partitioner(
            ResourcePatternResolver resourcePatternResolver) {
        MultiResourcePartitioner partitioner = new MultiResourcePartitioner();
        Resource[] resources;
        try {
            resources = resourcePatternResolver.getResources("file:"
                    + getFilesPath() + FILE_PATTERN);
        } catch (IOException e) {
            throw new RuntimeException(
                    "I/O problems when resolving the input file pattern.", e);
        }
        partitioner.setResources(resources);
        return partitioner;
    }

    @Bean
    public JobExecutionListener listener(ItemWriter<T> writer) {
        /* org.springframework.batch.core.scope.StepScope scope; */
        return new JobCompletionNotificationListener<T>(writer);
    }

    public abstract FieldSetMapper<T> getFieldSetMapper();

    public abstract String getFilesPath();

    public abstract String getLoaderName();

    public abstract String[] getColumnNames();

}

The JobScope for my writer() method results in the following exception :

    org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.writer': Scope 'job' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for job scope
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:353)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:195)
    at org.springframework.aop.target.SimpleBeanTargetSource.getTarget(SimpleBeanTargetSource.java:35)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:187)
    at $Proxy20.write(Unknown Source)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.writeItems(SimpleChunkProcessor.java:175)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.doWrite(SimpleChunkProcessor.java:151)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.write(SimpleChunkProcessor.java:274)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:199)
    at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:75)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330)
    at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
    at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:271)
    at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81)
    at org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate$ExecutingRunnable.run(TaskExecutorRepeatTemplate.java:262)
    at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalStateException: No context holder available for job scope
    at org.springframework.batch.core.scope.JobScope.getContext(JobScope.java:159)
    at org.springframework.batch.core.scope.JobScope.get(JobScope.java:92)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:338)
    ... 16 more
Feb 24, 2016 9:21:13 PM org.springframework.batch.core.step.AbstractStep execute
SEVERE: Encountered an error executing step LoadExposures ReadStep:slave in job LoadExposures
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.writer': Scope 'job' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for job scope
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:353)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:195)
    at org.springframework.aop.target.SimpleBeanTargetSource.getTarget(SimpleBeanTargetSource.java:35)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:187)
    at $Proxy20.write(Unknown Source)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.writeItems(SimpleChunkProcessor.java:175)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.doWrite(SimpleChunkProcessor.java:151)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.write(SimpleChunkProcessor.java:274)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:199)
    at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:75)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330)
    at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
    at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:271)
    at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81)
    at org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate$ExecutingRunnable.run(TaskExecutorRepeatTemplate.java:262)
    at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalStateException: No context holder available for job scope
    at org.springframework.batch.core.scope.JobScope.getContext(JobScope.java:159)
    at org.springframework.batch.core.scope.JobScope.get(JobScope.java:92)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:338)
    ... 16 more
Feb 24, 2016 9:21:13 PM org.springframework.batch.core.step.AbstractStep execute
SEVERE: Encountered an error executing step LoadExposures ReadStep:master in job LoadExposures
org.springframework.batch.core.JobExecutionException: Partition handler returned an unsuccessful step
    at org.springframework.batch.core.partition.support.PartitionStep.doExecute(PartitionStep.java:112)
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200)
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
    at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:392)
    at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135)
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306)
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135)
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
    at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
    at $Proxy15.run(Unknown Source)

When I get rid of the JobScope on the writer(), it works fine. Is my understanding of using JobScope incorrect?

Chetan Kinger
  • 15,069
  • 6
  • 45
  • 82
  • The accepted answer has resolved the issue. Had to make one more change. The writer method now needs to return the implementation class rather than `ItemWriter`. That is, it must return `ListItemWriter` – Chetan Kinger Feb 24 '16 at 16:45

1 Answers1

3

Your understanding of @JobScope seems correct.

I think the issue you are running into is https://jira.spring.io/browse/BATCH-2269

As referenced in that JIRA ticket, there is a workaround. Try replacing your TaskExecutor code with this:

@Bean
public TaskExecutor taskExecutor() {
    return new SimpleAsyncTaskExecutor() {
        @Override
        protected void doExecute(final Runnable task) {
            //gets the jobExecution of the configuration thread
            final JobExecution jobExecution = JobSynchronizationManager.getContext().getJobExecution();
            super.doExecute(new Runnable() {

                public void run() {
                    JobSynchronizationManager.register(jobExecution);

                    try {
                        task.run();
                    } finally {
                        JobSynchronizationManager.close();
                    }
                }
            });
        }
    };
}
Chetan Kinger
  • 15,069
  • 6
  • 45
  • 82
FGreg
  • 14,110
  • 10
  • 68
  • 110
  • Trying this out. JobExecution needs to be final. Also getting a compilation error saying "The method run() of type new Runnable(){} must override a superclass method". Also, `task` needs to be declared final. – Chetan Kinger Feb 24 '16 at 16:29
  • @CKing Sorry, the code in the ticket is Groovy I think so I was trying to convert it back to Java by hand. Missed a few things :). – FGreg Feb 24 '16 at 16:33
  • Thanks! This resolved the issue! Saved me a lot of time. :) Note that the writer method now needs to return the implementation class rather than `ItemWriter`. That is, it must return `ListItemWriter` due to some proxy issue which I still haven't understood completely. Also, this seems to make the AsyncTaskExecutor work like a SyncTaskExecutor. :( – Chetan Kinger Feb 24 '16 at 16:44