0

In Spring Batch I am looking to implement the retry mechanism. I have the steps configured below

This batch jobs actually runs and create the Payment csv files and place it to the outbound folder of my application server and through Tasklet I am calling the shellscript which picks all files and sends data to other server. While executing shell script through batch jobs, something fails, then I need to implement retry mechanism.

How Call Tasklet from another Tasklet with retry mechanism ?

public class ABCTasklet implements Tasklet, InitializingBean {

    @Value("${payment.directory}") // location where files are written - CSV
    private String location;

    @Override
    public RepeatStatus execute(StepContribution step, ChunkContext chunkContext) throws Exception {
        Set<String> payFileNames = new HashSet<>();
        List<FileDetails> fileStatusList = new ArrayList<>();

        File folder = new File(location);

        for (File file : folder.listFiles()) {
            FileDetails fileDetails = new FileDetails();
            fileDetails.setFileName(file.getName());
            fileDetails.setStatus(FileStatus.FAILURE.getStatus());
            fileStatusList.add(fileDetails);
            payFileNames.add(file.getName());
        }
        chunkContext.getStepContext().getStepExecution().getJobExecution().getExecutionContext().put("payFileNames", payFileNames);
        chunkContext.getStepContext().getStepExecution().getJobExecution().getExecutionContext().put("fileStatusList", fileStatusList);
        return RepeatStatus.FINISHED;
    }

    public String getLocation() {
        return location;
    }

    public void setLocation(String location) {
        this.location = location;
    }
}

Another Tasklet:

public class XYZListener implements Tasklet {
    @Autowired
    private RetryTemplate retryTemplate;

    private ExecutionContext executionContext;
    public ExecutionContext getExecutionContext() {
        return executionContext;
    }
    public void setExecutionContext(ExecutionContext executionContext) {
        this.executionContext = executionContext;
    }

    @Override
    public RepeatStatus execute(StepContribution paramStepContribution,ChunkContext paramChunkContext) throws Exception {
        Set<String> paymentFileNames = new HashSet<>();
        if(null != executionContext && null != executionContext.get("paymentFileNames")) {
            paymentFileNames = (Set<String>) executionContext.get("paymentFileNames");
        }

        try {
            for(String payFileName : paymentFileNames) {
                int exitStatus = 1;
                if (payFileName != null) {
    // execute shell scripts
                    ProcessBuilder procBuilder = new ProcessBuilder("/bin/bash","paymentBatchFiles.sh", payFileName);

                    Map<String, String> env = procBuilder.environment();
                    env.put("PATH", "/usr/local/bin:/bin:XXXXXXXXXXXXXXXXXX"); // many linux env. variables
                    env.put("MAYMENT_HOME","/Deployment/PaymentData"); 
                    ....
                    .....// other needed env variables

                    Process proc = procBuilder.start();
                    BufferedReader br = new BufferedReader(new InputStreamReader(proc.getErrorStream()));
                    exitStatus = proc.waitFor();
                }

                if (0 == exitStatus) {
                    paymentDetailsDAO.updateStatus(EDSProcessStatus.SENT.getStatusCode(), payFileName, JOB_NAME);
                }
                else {
                     // Implement Retry mechanism to call the ABCTasklet
                    retryTemplate.execute(); //TODO                  
                }
            }
            return RepeatStatus.FINISHED;
        } catch (InterruptedException | IOException ie) {

        }
    }
}

Job Details

<batch:decision id="paymentDecision" decider="paymentDecider">
    <batch:next on="FAILURE" to="failureStep" />
    <batch:next on="SUCCESS" to="x" />
</batch:decision>

<batch:step id="x">
    <batch:tasklet ref="payTasklet" />
    <batch:next on="COMPLETED" to="step1" />
</batch:step>

<batch:step id="step1">
    <batch:tasklet>
        <batch:chunk reader="paymentReader" writer="paymentWriter" commit-interval="100">
            <batch:listeners>
                <batch:listener ref="XYZListener" />
            </batch:listeners>
        </batch:chunk>
    </batch:tasklet>
    <batch:next on="COMPLETED" to="sendPaymentBatchFiles" />
</batch:step>

<batch:step id="sendPaymentBatchFiles">
    <batch:tasklet ref="sendPaymentBatchFilesTasklet">
    </batch:tasklet>
    <batch:next on="COMPLETED" to="lastRunUpdate" />
</batch:step>

<batch:step id="lastRunUpdate" next="Z">
    <batch:tasklet ref="QQQQRunUpdater" />
</batch:step>


<batch:step id="Z">
    <batch:tasklet ref="PPTasklet" />
</batch:step>

<batch:step id="failureStep" next="sendPaymentBatchFiles">
    <batch:tasklet ref="ABCTasklet" />
</batch:step>

How to call retry mechanism i.e ABCTasklet class from XYZListener ? Code update will be appreciated

EDIT-1 I did below, but doesn't seems to be working ?

return retryTemplate.execute(new RetryCallback<RepeatStatus, RuntimeException>() {
    @Override
    public RepeatStatus doWithRetry(RetryContext context) {
        logger.info("Retry File, Retry Count = "+context.getRetryCount());
        return executeShellScript();
    }
});

Edit-2:

RepeatTemplate template = new RepeatTemplate();
template.setCompletionPolicy(new DefaultResultCompletionPolicy());

StringRepeatCallback callback = new StringRepeatCallback();
callback.setBillingFileNames(billingFileNames);
template.iterate(callback);

Another class:

public class StringRepeatCallback implements RepeatCallback{
    @Autowired
    private PaymentFilesTasklet tasklet;

    private Set<String> paymentFileNames;

    public void setBillingFileNames(Set<String> paymentFileNames) {
        this.paymentFileNames = paymentFileNames;
    }

    @Override
    public RepeatStatus doInIteration(RepeatContext repeatContext) throws Exception {
        tasklet.executeShellScript(paymentFileNames);
        return RepeatStatus.CONTINUABLE;
    }
}

The error which I am getting is:

2018-07-22 13:12:28,287 ERROR [main] (AbstractStep.java:229) - Encountered an error executing step sendPaymentBatchFiles in job paymentBatchJob
java.lang.NullPointerException
        at com.mastercard.ess.eds.billing.callback.StringRepeatCallback.doInIteration(StringRepeatCallback.java:24)
        at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374)
        at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
        at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
        at com.mastercard.ess.eds.billing.tasklet.SendBillingBatchFilesTasklet.executeShellScript(SendBillingBatchFilesTasklet.java:132)
        at com.mastercard.ess.eds.billing.tasklet.SendBillingBatchFilesTasklet.execute(SendBillingBatchFilesTasklet.java:152)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
        at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
        at com.sun.proxy.$Proxy44.execute(Unknown Source)
        at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406)
        at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
        at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:271)
        at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81)
        at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374)
        at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
        at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
        at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257)
        at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200)
        at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
        at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64)
        at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67)
        at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169)
        at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144)
        at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:134)
        at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306)
        at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135)
        at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
        at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
        at com.sun.proxy.$Proxy34.run(Unknown Source)
        at org.springframework.batch.core.launch.support.CommandLineJobRunner.start(CommandLineJobRunner.java:362)
        at org.springframework.batch.core.launch.support.CommandLineJobRunner.main(CommandLineJobRunner.java:592)
2018-07-22 13:12:28,459 ERROR [main] (AbstractJob.java:335) - Encountered fatal error executing job
Jeff Cook
  • 7,956
  • 36
  • 115
  • 186
  • It is not clear why you need to call a Tasklet from another Tasklet but can you show the code you use to do that? We will then be able to see if it is possible to add retry on it.. – Mahmoud Ben Hassine Jul 20 '18 at 10:23
  • Ok I see. This is not calling a tasklet from another tasklet. What you can do is create a flow that points to ABCTasklet when XYZListener (which is a tasklet and not a listener but the naming is up to you) fails. Another option I can suggest (since I see you want to use a `RetryTemplate` in your `XYZListener`) is to wrap the code that may fail in a `RetryCallback`. See example here: https://github.com/spring-projects/spring-retry#recoverycallback. – Mahmoud Ben Hassine Jul 20 '18 at 19:49
  • Probably you don't need a retry template. You can return `RepeatStatus.CONTINUABLE` if the invocation fails (and increment a variable retryCount) and then throw an exception if the retry count is exceeded. But I would recommend creating a flow of execution and re-execute the appropriate step on failure. – Mahmoud Ben Hassine Jul 21 '18 at 16:26
  • @@ Mahmoud Ben Hassine- The above logic I implemented above is executing infinite times, not sure why this is happening. Could you please guide ASAP ? – Jeff Cook Jul 21 '18 at 21:48
  • The tasklet will be re-executed each time your return `RepeatStatus.CONTINUABLE` until your return `RepeatStatus.FINISHED` to signal success or your throw an exception to signal failure: https://docs.spring.io/spring-batch/4.0.x/reference/html/step.html#taskletStep In your case, you can create a counter in your tasklet implementation called `retryCount`. When you call your script and it fails, you increment the counter and return `CONTINUABLE`. If the counter exceed the max retries, you throw an exception. Otherwise, you return `FINISHED`. I still recommend creating a flow for your use case – Mahmoud Ben Hassine Jul 23 '18 at 05:12

0 Answers0