Log read, write and error counts with Spring batch

Question

I have a Sprint batch with the job as

<batch:job id="data-send" parent="baseJob">
    <batch:listeners merge="true">
        <batch:listener ref="someJobExecutionListener"/>
    </batch:listeners>
    <batch:step id="data-send.step01">
        <batch:tasklet transaction-manager="jobTransactionManager" ref="dataSendTasklet"/>
    </batch:step>
</batch:job>

The tasklet pseudo code is

@Component
public class DataSendTasklet implements Tasklet {
 
@Override
public RepeatStatus execute((final StepContribution contribution, final ChunkContext chunkContext) throws Exception {

   List<SomeObject> objectList = dbGetOperation();
   objectList.stream().forEach(o -> someDataSendOperation(o));
   log("ReadCount: "+objectList.size());
   return RepeatStatus.FINISHED;
}

Currently, I am logging the read count by calculating the objectList's size and If I had to log the processed records and error records I would have to keep track of a variable inside the someDataSendOperation method. Is there any Spring off-the-shelf functionality to handle the read, write and error counts in this scenario?

score 1 · Answer 1 · answered Aug 03 '20 at 20:00

I had a similar use case, I created a StepListener and print all the useful information. Here is the example:

public class StepExecuteListener extends StepExecutionListenerSupport {
private static final Logger LOG = LoggerFactory.getLogger(StepExecuteListener.class);

@Override
public void beforeStep(StepExecution stepExecution) {
    // No action needed
}

@Override
public ExitStatus afterStep(StepExecution stepExecution) {
    
    LOG.debug("StepExecutionListener - afterStep:getCommitCount= {}", stepExecution.getCommitCount());
    LOG.debug("StepExecutionListener - afterStep:getFilterCount= {}", stepExecution.getFilterCount());
    LOG.debug("StepExecutionListener - afterStep:getProcessSkipCount= {}", stepExecution.getProcessSkipCount());
    LOG.info("StepExecutionListener - afterStep:getReadCount= {}", stepExecution.getReadCount());
    LOG.debug("StepExecutionListener - afterStep:getReadSkipCount= {}", stepExecution.getReadSkipCount());
    LOG.debug("StepExecutionListener - afterStep:getRollbackCount= {}", stepExecution.getRollbackCount());
    LOG.info("StepExecutionListener - afterStep:getWriteCount= {}", stepExecution.getWriteCount());
    LOG.debug("StepExecutionListener - afterStep:getWriteSkipCount= {}", stepExecution.getWriteSkipCount());
    LOG.debug("StepExecutionListener - afterStep:getStepName= {}", stepExecution.getStepName());
    LOG.debug("StepExecutionListener - afterStep:getSummary= {}", stepExecution.getSummary());
    LOG.debug("StepExecutionListener - afterStep:getStartTime= {}", stepExecution.getStartTime());
    LOG.debug("StepExecutionListener - afterStep:getStartTime= {}", stepExecution.getEndTime());
    LOG.debug("StepExecutionListener - afterStep:getLastUpdated= {}", stepExecution.getLastUpdated());
    LOG.debug("StepExecutionListener - afterStep:getExitStatus= {}", stepExecution.getExitStatus());
    LOG.debug("StepExecutionListener - afterStep:getFailureExceptions= {}", stepExecution.getFailureExceptions());
   
    
    return null;
}
   }

Hi, since in my use case I don't have a reader or writer beans the values in stepExecution are not correct. The readCount I get when I tried is 0 — Sooraj, Aug 04 '20 at 02:34

score 0 · Answer 2 · answered Aug 14 '20 at 09:31

I don't see why you use a simple tasklet instead of using a chunk-oriented step. You could use an ItemReader<SomeObject> and an ItemWriter<SomeObject> that does the someDataSendOperation.

With that, you don't need to iterate over the list yourself and you will get the counts you are looking for.

Log read, write and error counts with Spring batch

2 Answers2