Spring Batch custom writer close method getting called twice (requiring special handling)

Question

I have the following writer class prototype:

public class MyWriter extends AbstractItemStreamItemWriter<FieldSet> {
    ...
    @Override
    public void close()
    {
        ...
    {
    ...
}

My Job is defined by these @Beans:

    @Bean
    protected Step step(ItemReader<FieldSet> reader, ItemWriter<FieldSet> writer)
    {
        return stepBuilder.get("step")
            .<FieldSet, FieldSet> chunk(chunkSize)
            .reader(reader)
            .writer(writer)
            .listener(this)
            .build();
    }

    @Bean
    protected Job myImportJob(Step step, JobExecutionListener jobExecutionListener)
    {
        return jobBuilder
            .get("myImportJob")
            .listener(jobExecutionListener)
            .start(step)
            .build();
    }

The job is triggered from a MQ listener like this:

@Autowired
private Job job;

@Autowired
private JobLauncher jobLauncher;

@JmsListener(destination = "queue_name")
public void receiveMessage(TextMessage message) throws JMSException
{
    log.warn("Received message {} with listener {}.", message, this);

    JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();

    // add job parameters here
    ...

    JobParameters jobParameters = jobParametersBuilder.toJobParameters();

    try
    {
        JobExecution jobExecution = jobLauncher.run(job, jobParameters);
        log.trace("jobExecution = " + jobExecution);
    }
    catch (JobExecutionException jee)
    {
        log.warn("Could not run job {}", job);
    }
}

The problem I'm encountering is that MyWriter's close method gets called twice. The stack traces for each call are:

Thread [DefaultMessageListenerContainer-2] (Suspended (breakpoint at line 162 in MyWriter)) 
    MyWriter.close() line: 162  
    MyWriter$$FastClassBySpringCGLIB$$80508a22.invoke(int, Object, Object[]) line: not available    
    MethodProxy.invoke(Object, Object[]) line: 204  
    CglibAopProxy$CglibMethodInvocation.invokeJoinpoint() line: 720 
    CglibAopProxy$CglibMethodInvocation(ReflectiveMethodInvocation).proceed() line: 157 
    DelegatingIntroductionInterceptor.doProceed(MethodInvocation) line: 133 
    DelegatingIntroductionInterceptor.invoke(MethodInvocation) line: 121    
    CglibAopProxy$CglibMethodInvocation(ReflectiveMethodInvocation).proceed() line: 179 
    CglibAopProxy$DynamicAdvisedInterceptor.intercept(Object, Method, Object[], MethodProxy) line: 655  
    MyWriter$$EnhancerBySpringCGLIB$$e345242b.close() line: not available   
    CompositeItemStream.close() line: 85    
    TaskletStep.close(ExecutionContext) line: 305   
    TaskletStep(AbstractStep).execute(StepExecution) line: 271  
--> SimpleStepHandler.handleStep(Step, JobExecution) line: 148  
    SimpleJob(AbstractJob).handleStep(Step, JobExecution) line: 392 
    SimpleJob.doExecute(JobExecution) line: 135 
    SimpleJob(AbstractJob).execute(JobExecution) line: 306  
    SimpleJobLauncher$1.run() line: 135 
    SyncTaskExecutor.execute(Runnable) line: 50 
    SimpleJobLauncher.run(Job, JobParameters) line: 128 
    NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]  
    NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62  
    DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43  
    Method.invoke(Object, Object...) line: 497  
    AopUtils.invokeJoinpointUsingReflection(Object, Method, Object[]) line: 302 
    ReflectiveMethodInvocation.invokeJoinpoint() line: 190  
    ReflectiveMethodInvocation.proceed() line: 157  
    SimpleBatchConfiguration$PassthruAdvice.invoke(MethodInvocation) line: 127  
    ReflectiveMethodInvocation.proceed() line: 179  
    JdkDynamicAopProxy.invoke(Object, Method, Object[]) line: 208   
    $Proxy105.run(Job, JobParameters) line: not available   
    MyMQListener$$EnhancerBySpringCGLIB$$28277c0e(MyMQListener).receiveMessage(TextMessage) line: 74    
    NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]  
    NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62  
    DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43  
    Method.invoke(Object, Object...) line: 497  
    InvocableHandlerMethod.doInvoke(Object...) line: 198    
    InvocableHandlerMethod.invoke(Message<?>, Object...) line: 116  
    MessagingMessageListenerAdapter.invokeHandler(Message, Session, Message<?>) line: 90    
    MessagingMessageListenerAdapter.onMessage(Message, Session) line: 66    
    DefaultMessageListenerContainer(AbstractMessageListenerContainer).doInvokeListener(SessionAwareMessageListener, Session, Message) line: 721 
    DefaultMessageListenerContainer(AbstractMessageListenerContainer).invokeListener(Session, Message) line: 681    
    DefaultMessageListenerContainer(AbstractMessageListenerContainer).doExecuteListener(Session, Message) line: 651 
    DefaultMessageListenerContainer(AbstractPollingMessageListenerContainer).doReceiveAndExecute(Object, Session, MessageConsumer, TransactionStatus) line: 315 
    DefaultMessageListenerContainer(AbstractPollingMessageListenerContainer).receiveAndExecute(Object, Session, MessageConsumer) line: 253  
    DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener() line: 1150 
    DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop() line: 1142 
    DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run() line: 1039    
    Thread.run() line: 745

and

Thread [DefaultMessageListenerContainer-2] (Suspended (breakpoint at line 162 in MyWriter)) 
    MyWriter.close() line: 162  
    NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]  
    NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62  
    DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43  
    Method.invoke(Object, Object...) line: 497  
    DisposableBeanAdapter.invokeCustomDestroyMethod(Method) line: 354   
    DisposableBeanAdapter.destroy() line: 277   
    DisposableBeanAdapter.run() line: 236   
    StepContext.close() line: 213   
    StepSynchronizationManager$1.close(StepContext) line: 53    
    StepSynchronizationManager$1.close(Object) line: 36 
    StepSynchronizationManager$1(SynchronizationManagerSupport<E,C>).release() line: 190    
    StepSynchronizationManager.release() line: 112  
    TaskletStep(AbstractStep).doExecutionRelease() line: 290    
    TaskletStep(AbstractStep).execute(StepExecution) line: 278  
--> SimpleStepHandler.handleStep(Step, JobExecution) line: 148  
    SimpleJob(AbstractJob).handleStep(Step, JobExecution) line: 392 
    SimpleJob.doExecute(JobExecution) line: 135 
    SimpleJob(AbstractJob).execute(JobExecution) line: 306  
    SimpleJobLauncher$1.run() line: 135 
    SyncTaskExecutor.execute(Runnable) line: 50 
    SimpleJobLauncher.run(Job, JobParameters) line: 128 
    NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]  
    NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62  
    DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43  
    Method.invoke(Object, Object...) line: 497  
    AopUtils.invokeJoinpointUsingReflection(Object, Method, Object[]) line: 302 
    ReflectiveMethodInvocation.invokeJoinpoint() line: 190  
    ReflectiveMethodInvocation.proceed() line: 157  
    SimpleBatchConfiguration$PassthruAdvice.invoke(MethodInvocation) line: 127  
    ReflectiveMethodInvocation.proceed() line: 179  
    JdkDynamicAopProxy.invoke(Object, Method, Object[]) line: 208   
    $Proxy105.run(Job, JobParameters) line: not available   
    MyMQListener$$EnhancerBySpringCGLIB$$28277c0e(MyMQListener).receiveMessage(TextMessage) line: 74    
    NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]  
    NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62  
    DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43  
    Method.invoke(Object, Object...) line: 497  
    InvocableHandlerMethod.doInvoke(Object...) line: 198    
    InvocableHandlerMethod.invoke(Message<?>, Object...) line: 116  
    MessagingMessageListenerAdapter.invokeHandler(Message, Session, Message<?>) line: 90    
    MessagingMessageListenerAdapter.onMessage(Message, Session) line: 66    
    DefaultMessageListenerContainer(AbstractMessageListenerContainer).doInvokeListener(SessionAwareMessageListener, Session, Message) line: 721 
    DefaultMessageListenerContainer(AbstractMessageListenerContainer).invokeListener(Session, Message) line: 681    
    DefaultMessageListenerContainer(AbstractMessageListenerContainer).doExecuteListener(Session, Message) line: 651 
    DefaultMessageListenerContainer(AbstractPollingMessageListenerContainer).doReceiveAndExecute(Object, Session, MessageConsumer, TransactionStatus) line: 315 
    DefaultMessageListenerContainer(AbstractPollingMessageListenerContainer).receiveAndExecute(Object, Session, MessageConsumer) line: 253  
    DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener() line: 1150 
    DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop() line: 1142 
    DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run() line: 1039    
    Thread.run() line: 745

The last common line in the stack is SimpleStepHandler.handleStep(Step, JobExecution) line: 148, marked with arrows.

I currently have a special class-level boolean variable that only exists to make sure the guts of the close method only run once, but this seems not quite right to me.

There are no exceptions or problems with reading or writing. This is not due to Skip or Restart functionality.

How can I keep the ItemWriter.close() method from getting called twice?

please post more infos, how do start the jobs? it looks rather strange to see cglib work there, did you configure any AOP functions? — Michael Pralow, Feb 16 '16 at 08:19
I added the code that launches the job. Is this all you need? — arnaldop, Feb 16 '16 at 22:02
wrong guess from my side, cglib proxies are needed for the stepscoped objects, i debugged here a simple itemwriter example, close is called only once, is your writer used elsewhere? is the job running twice? — Michael Pralow, Feb 17 '16 at 07:26
Found the problem. The way I was handling StepExecution was causing my writer object to reuse StepExecutions, clobbering data. This was also causing some writer objects to have the closed method invoked twice. Not sure why the stack trace is different, but once I fixed the reuse problem, the close problem went away. Thanks! — arnaldop, Feb 17 '16 at 14:47
Could you please clarify your answer? What exactly did you fix regarding StepExecution and how you were handling it? I'm running into the same problem but I haven't defined any StepExecution related beans / configuration. Only similarity is that we're both extending from AbstractItemStreamItemWriter — Zachary Carter, Aug 24 '16 at 17:15
Unfortunately, Zachary, I don't remember exactly what I did, and no longer have access to the codebase. One thing I do remember that is not included in my answer is that the way I was defining my workflow, I did not have a way to "stop", or signal that I had reached the end. Therefore, it kept looping over the same data. I had to know from my input stream that I had no more data and let the StepExecution that it was done. Hope this helps! — arnaldop, Nov 06 '16 at 20:35

score 8 · Answer 1 · answered Oct 04 '18 at 08:49

I had a terrible headache because of this error.

Spring tries to automatically infer a destroyMethod when using Java configuration (but it does not do so when using XML configuration). To disable this automatic inference, use:

 @Bean(destroyMethod="")

Now using this annotation Spring stop calling my method twice and it will only be called once by Spring batch.

score 0 · Answer 2 · answered Dec 13 '22 at 13:39

0

In addition to the destroyMethod="" trick, if you have several methods producing beans of the same type, you can have that type implement DisposableBean and include an empty destroy() method to avoid close() being called.

answered Dec 13 '22 at 13:39

gizmo71

41
4

Jim Cox · Answer 3 · 2023-03-16T19:29:11.587

0

It's being called twice because you are missing @StepScope on your step bean.

edited Mar 16 '23 at 19:29

answered Mar 15 '23 at 18:23

Jim Cox

974
7
10

Spring Batch custom writer close method getting called twice (requiring special handling)

3 Answers3