Using Spring Batch to read to convert rows from input CSV file to output CSV file with 1 to many relation

Question

I've already published this question: How to use Spring Batch read CSV,process it and write it as a CSV with one row can produce more than one row?

And also reviewed these relevant answers: Spring Batch - Using an ItemWriter with List of Lists

But still can't figure out how to use Spring Batch in order to:

1. Read a row from input CSV file.
2. Process it and produce one or more output rows.
3. Write the output rows into the output file.

I know that the solution should be implementing a writer that will accepts a list of items and will use "delegate" in some way in order to process the items one by one.

I would appreciate if someone can shed some light on this.

My code:

public class CsvRowsProcessor implements ItemProcessor<RowInput, List<RowOutput>>{

    @Override
    public List<RowOutput> process(final RowInput rowInput)  {

        final String id = rowInput.getId();
        final String title = rowInput.getTitle();
        final String description = rowInput.getDescription();
        final RowOutput transformedRowInput = new RowOutput(id, title, description);

        List<RowOutput> rows=new LinkedList<>();
        rows.add(transformedRowInput);
        return rows;
    }

}

@Bean
ItemWriter<RowOutput> csvRowsWriter() {
    FlatFileItemWriter<RowOutput> csvFileWriter = new FlatFileItemWriter<>();
    csvFileWriter.setResource(new FileSystemResource("C:\\Users\\orenl\\IdeaProjects\\Spring-Batch-CSV-Example\\src\\main\\resources\\outputFile.csv"));
    LineAggregator<RowOutput> lineAggregator = createLineAggregator();
    csvFileWriter.setLineAggregator(lineAggregator);
    csvFileWriter.setHeaderCallback(new FlatFileHeaderCallback() {

        public void writeHeader(Writer writer) throws IOException {
            writer.write("Id,Title,Description");
        }
    });
    return csvFileWriter;
}



private LineAggregator<RowOutput> createLineAggregator() {
    DelimitedLineAggregator<RowOutput> lineAggregator = new DelimitedLineAggregator<>();
    lineAggregator.setDelimiter(",");

    FieldExtractor<RowOutput> fieldExtractor = createFieldExtractor();
    lineAggregator.setFieldExtractor(fieldExtractor);

    return lineAggregator;
}

private FieldExtractor<RowOutput> createFieldExtractor() {
    BeanWrapperFieldExtractor<RowOutput> extractor = new BeanWrapperFieldExtractor<>();
    extractor.setNames(new String[] { "Id", "Title", "Description" });
    return extractor;
}

@Bean
public Step csvFileToFileStep() {
    return stepBuilderFactory.get("csvFileToFileStep")
            .<RowInput ,RowOutput>chunk(1)
            .reader(csvRowsReader())
            .processor(csvRowsProcessor())
            .writer(csvRowsWriter())
            .build();
}

@Bean
Job csvFileToCsvJob(JobCompletionNotificationListener listener) {
    return jobBuilderFactory.get("csvFileToCsvJob")
            .incrementer(new RunIdIncrementer())
            .listener(listener)
            .flow(csvFileToFileStep())
            .end()
            .build();
}

Answer 1

Here is an example:

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.support.ListItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableBatchProcessing
public class MyJob {

    @Autowired
    private JobBuilderFactory jobs;

    @Autowired
    private StepBuilderFactory steps;

    @Bean
    public ItemReader<Integer> itemReader() {
        return new ListItemReader<>(Arrays.asList(1, 3, 5, 7, 9));
    }

    @Bean
    public ItemProcessor<Integer, List<Integer>> itemProcessor() {
        return item -> {
            List<Integer> result = new ArrayList<>();
            result.add(item);
            result.add(item + 1);
            return result;
        };
    }

    @Bean
    public ItemWriter<List<Integer>> itemWriter() {
        return items -> {
            for (List<Integer> item : items) {
                for (Integer integer : item) {
                    System.out.println("integer = " + integer);
                }
            }
        };
    }

    @Bean
    public Step step() {
        return steps.get("step")
                .<Integer, List<Integer>>chunk(2)
                .reader(itemReader())
                .processor(itemProcessor())
                .writer(itemWriter())
                .build();
    }

    @Bean
    public Job job() {
        return jobs.get("job")
                .start(step())
                .build();
    }

    public static void main(String[] args) throws Exception {
        ApplicationContext context = new AnnotationConfigApplicationContext(MyJob.class);
        JobLauncher jobLauncher = context.getBean(JobLauncher.class);
        Job job = context.getBean(Job.class);
        jobLauncher.run(job, new JobParameters());
    }

}

This sample reads some numbers and for each number it returns the number and its successor and then prints numbers to the standard output. The example shows how the processing of one item returns multiple items.

It prints:

integer = 1
integer = 2
integer = 3
integer = 4
integer = 5
integer = 6
integer = 7
integer = 8
integer = 9
integer = 10

You can adapt the sample to read/write from/to files.

Hope this helps.