FlatFileItemWriterBuilder-headerCallback() get number of rows written

Question

Is it possible to get the total number of rows written from FlatFileItemWriter.headerCallback()?

I am a spring-batch nubee and I looked at putting count of lines into header of flat file and Spring Batch - Counting Processed Rows. However I can't seem to implement the logic using the advice given there. It makes sense the writer count will only be available after the file is processed. However I am trying to get the row-count just before the file is officially written.

I tried to look for a hook like @AfterStep and grab the total rows, but I keep going in circles.

@Bean
@StepScope
public FlatFileItemWriter<MyFile> generateMyFileWriter(Long jobId,Date eventDate) {
    String filePath = "C:\MYFILE\COMPLETED";
    Resource file = new FileSystemResource(filePath);

    DelimitedLineAggregator<MyFile> myFileLineAggregator = new DelimitedLineAggregator<>();
    myFileLineAggregator.setDelimiter(",");
    myFileLineAggregator.setFieldExtractor(getMyFileFieldExtractor());

    return new FlatFileItemWriterBuilder<MyFile>()
            .name("my-file-writer")
            .resource(file)
            .headerCallback(new MyFileHeaderWriter(file.getFilename()))
            .lineAggregator(myFileLineAggregator)
            .build();
}

private FieldExtractor<MyFile> getMyFileFieldExtractor() {
    final String[] fieldNames = new String[]{
            "typeRecord",
            "idSystem"               
    };  
    return item -> {
        BeanWrapperFieldExtractor<MyFile> extractor = new BeanWrapperFieldExtractor<>();
        extractor.setNames(fieldNames);
        return extractor.extract(item);
    };
}

Notice I am using the MyFileHeaderWriter.java class(below) in the headerCallback(new MyFileHeaderWriter(file.getFilename())) (above). I am trying to initialize the value of qtyRecordsCreated below.

class MyFileHeaderWriter implements FlatFileHeaderCallback {

 private final String header;

 private String dtxCreated;
 private String tmxCreated;
 private String fileName;//15 byte file name    private String qtyRecordsCreated;//number of rows in file including the header row

 MyFileHeaderWriter(String sbfFileName) {
    SimpleDateFormat dateCreated = new SimpleDateFormat("YYDDD");
    SimpleDateFormat timeCreated = new SimpleDateFormat("HHMM");
    Date now = new Date(); 

    this.dtxCreated = dateCreated.format(now);
    this.tmxCreated = timeCreated.format(now);
    this.fileName = sbfFileName;        this.qtyRecordsCreated="";  

    String[] headerValues = {dtxCreated,tmxCreated,fileName,qtyRecordsCreated};
     this.header = String.join(",", headerValues);
 }

 @Override
 public void writeHeader(Writer writer) throws IOException {
     writer.write(header);
 }
}

How can I get the number of rows in the header row? Can the FlatFileFooterCallback be used to fetch the number of rows and then update the header with number of rows in the file afterwards?

`update the header with number of rows in the file afterwards`: you can't insert data in a file inline afterwards. You need to find a way to calculate the number of records upfront and insert it in the header (like a `select count` query, or `Files.lines("yourfile").count()`, etc) — Mahmoud Ben Hassine, Jan 29 '20 at 09:34

A.khalifa · Answer 1 · 2020-02-03T08:00:24.843

0

You can achieve this in ItemProcessor, try this it work for me

public class EmployeeProcessor implements ItemProcessor<Employee, Employee> {
    @Override
    public Employee process(Employee employee) throws Exception {

         return employee;
    }

     @AfterStep
    public void afterStep(StepExecution stepExecution) {
      ExecutionContext stepContext = stepExecution.getExecutionContext();
      stepContext.put("count",  stepExecution.getReadCount());
      System.out.println("COUNT" + stepExecution.getReadCount());
    }
}

And in you writer to get value

int count = stepContext.getInt("count");

Hope work for you

edited Feb 03 '20 at 08:00

answered Jan 29 '20 at 15:44

A.khalifa

2,366
3
27
40

How can I access the value when setting up the writer? Is there a context where I can store the value and pull it when I use the FlatFileItemWriterBuilder.headerCallback – Faiyet Jan 31 '20 at 17:29
@Faiyet chek this also https://docs.spring.io/spring-batch/docs/current/reference/html/common-patterns.html – A.khalifa Feb 03 '20 at 13:35

FlatFileItemWriterBuilder-headerCallback() get number of rows written

1 Answers1