How can I get the number of line and the filename in itemProcessor - spring batch

Question

I am using spring batch to parse my files. In ItemProcessor I validate if the incoming fields are correct. If they are not I want to throw a ValidationException and log to a file the corresponding row which has the incorrect fields. So, how can I find the number of line and the filename in ItemProcessor?

BenG · Answer 1 · 2013-03-04T09:24:34.300

Without seeing you ItemReader config I can't really be sure but if you are using something like FlatFileItemReader to parse a csv, if in strict mode it will validate the number of columns.

Assuming you reader looks like this, that is:

<bean id="iItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
    <property name="linesToSkip" value="1"/>
    <property name="comments" value="#" />
    <property name="encoding" value="UTF-8"/>
    <property name="lineMapper" >
        <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
            <property name="lineTokenizer">
                <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                    <property name="delimiter" value=","/>
                    <property name="names">
                        <list >
                            <value>First_Field</value>
                        <value>Second_Field</value>
                        </list>
                    </property>
                    <property name="strict" value="true"/>
                </bean>
            </property>
            <property name="fieldSetMapper">
                <bean class="uk.co.package.FieldSetMapper">
                    <property name="dateFormat" value="yyyy-MM-dd HH:mm:ss"/>
                </bean>
            </property>
        </bean>
    </property>

</bean>

It will throw a FlatFileParseException for any lines that can't be processed. This includes the line number and can be handled in a listener.

Thank you for your answer! I use this configuration to catch the exceptions that will be occurred from flatFileItemReader. Ηowever, the validation of the fields of rows is performed from itemProcessor. For example in itemProcessor I check if the first string consist of 10 chars, the second field is a string which belongs in a specific list of string. That you suggested to me is to transfer the validation in fieldSetMapper during the creation of the object from the row in order to catch the flatFileItemReaderException? — pbal, Mar 06 '13 at 12:47
I was suggesting to move your validation into the fieldSetMapper, but another approach could be to count the lines in the ItemProcessor (assuming its step scoped) and store the filename in the executioncontext so its available for the whole job. — BenG, Mar 26 '13 at 13:54

score 0 · Answer 2 · edited May 23 '17 at 12:08

As for the line number, you might build your own LineMapper and then store the line-number in your business object. An example in which I store the line unprocessed (as-is) together with the line number:

    DefaultLineMapper<OneRow> lineMapper = new DefaultLineMapper<OneRow>() {
        @Override
        public OneRow mapLine(String line, int lineNumber) throws Exception {
            return new OneRow(lineNumber, line);
        }
    };

Of course you can already map your Object, I had the need to have the whole line unprocessed as input to my Processors.

As a reference with the same idea: https://stackoverflow.com/a/23770421/5658642

How can I get the number of line and the filename in itemProcessor - spring batch

2 Answers2