spring batch ItemReader FlatFileItemReader set cursor to start reading from a particular line or set linestoskip dynamically

Question

In my springbatch+quartz setup, I am reading a CSV File using FlatFileItemReader. I want to set the cursor for the reader to start the next jobinstance with the given parameters for reader. Is it possible?

<bean id="cvsFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
    <!-- Read a csv file -->
    <property name="resource" value="classpath:cvs/input/report.csv" />

    <property name="lineMapper">
        <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
            <property name="lineTokenizer">
                <bean
                    class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                    <property name="names" value="id,impressions" />
                </bean>
            </property>
            <property name="fieldSetMapper">
                <bean
                    class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
                    <property name="prototypeBeanName" value="report" />
                </bean>
            </property>
        </bean>
    </property>

</bean>

The idea is to continue reading the file where last failure occured in the next execution. I am putting an integer 'writecursor' for each line written in my customWriter.

public void write(List<? extends Report> items) throws Exception {

System.out.println("writer..." + items.size() + " > ");     
for(Report item : items){
    System.out.println("writing item id: " + item.getId());     
    System.out.println(item);

}
//getting stepExecution by implementing StepExecutionListener
this.stepExecution.getExecutionContext().putInt("writecursor", ++writecursor);

}

Now, in the customItemReadListener, I want to get the update writecursor value and then skip the lines from the top to start reading from writecursor

public class CustomItemReaderListener implements ItemReadListener<Report>, StepExecutionListener {

    ApplicationContext context = ApplicationContextUtils.getApplicationContext();
    private StepExecution stepExecution;
    @Override
    public void beforeRead() {
        //Skip lines somehow
    }

Another thing I saw as a possible solution is to set linestoskip dynamically in itemreader. There is a thread here http://thisisurl.com/dynamic-value-for-linestoskip-in-itemreader but not answered yet. And here, http://forum.spring.io/forum/spring-projects/batch/100950-accessing-job-execution-context-from-itemwriter

possible duplicate of [Spring batch job csv file read recover from the line where it failed](http://stackoverflow.com/questions/21451934/spring-batch-job-csv-file-read-recover-from-the-line-where-it-failed) — Luca Basso Ricci, Jan 31 '14 at 10:25
Yeah, possible duplicate. I tried using linestoskip property of reader. But now I think, intead of storing the cursor value in stepexecution, I need to store it in jobexecutioncontext as the cursor value is reset in the next jobexecution. How to access jobexecutioncontext in ItemReader/Processor/Writer? — UserBSS1, Jan 31 '14 at 10:40
Check my answer. But IMHO you are only replicating an SB built-in feature — Luca Basso Ricci, Jan 31 '14 at 10:54
I am kind of trying to make a custom spring batch admin with quartz so needs to override different scenarios and insert my own logic — UserBSS1, Jan 31 '14 at 12:34

score 6 · Accepted Answer · answered Jan 31 '14 at 10:22

6

Use FlatFileItemReader.linesToSkip property setted injecting job Parameter value.

<bean id="myReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
    <property name="linesToSkip" value="file:#{jobParameters['cursor']}" />
</bean>

answered Jan 31 '14 at 10:22

Luca Basso Ricci

17,829
2
47
69

its working fine for me. Can jobParamertes be replaced with stepExecutionContext or jobExecutionContext as well? – UserBSS1 Jan 31 '14 at 12:31
yes you can wire it up to use stepExecution or jobExecution data instead. – Jackson Ha Jan 31 '14 at 13:20

score 2 · Answer 2 · edited Feb 11 '15 at 20:42

A more easy way for implementing the lines to skip is by the following:

create a reader flat file reader in the xml, autowire the reader to the beforeStep of step execution listener as shown below

public class CustomStepListener implements StepExecutionListener {

    @Autowired
    @Qualifier("cvsFileItemReader")
    FlatFileItemReader cvsFileItmRdr;

    @Override
    public void  beforeStep(StepExecution stepExecution) {
        System.out.println("StepExecutionListener -------- beforeStep");            
        cvsFileItmRdr.setLinesToSkip(4);
    }

    @Override
    public ExitStatus afterStep(StepExecution stepExecution) {
        System.out.println("StepExecutionListener - afterStep");
        return null;
    }

}

it is working fine for me.....

spring batch ItemReader FlatFileItemReader set cursor to start reading from a particular line or set linestoskip dynamically

2 Answers2

Linked