0

I tried to find a solution for the problem I am looking for, but I dint find it or a chance that I might skipped. Help me if you can redirect me to the solution page.

Input to the batch: I have TRADES.csv, PORTFOLIO.csv files which are located at different paths.

How I am implemented it currently: Currently I wrote a class CSVReader has two FlatFileItemReaders which are defined as below.

<beans:bean id="porfolioReader"
    class="org.springframework.batch.item.file.FlatFileItemReader">
    <beans:property name="lineMapper" ref="lineMapperForPortfolio"></beans:property>
    <beans:property name="strict" value="false"></beans:property>
    <beans:property name="recordSeparatorPolicy"
        ref="csvRecordSeparatorPolicy"></beans:property>
    <beans:property name="linesToSkip" value="1"></beans:property>
    <beans:property name="encoding" value="ISO-8859-1"></beans:property>
</beans:bean>

<beans:bean id="tradeReader"
    class="org.springframework.batch.item.file.FlatFileItemReader">
    <beans:property name="lineMapper" ref="lineMapperForTrades"></beans:property>
    <beans:property name="strict" value="false"></beans:property>
    <beans:property name="recordSeparatorPolicy"
        ref="csvRecordSeparatorPolicy"></beans:property>
    <beans:property name="linesToSkip" value="1"></beans:property>
    <beans:property name="encoding" value="ISO-8859-1"></beans:property>
</beans:bean>

So based on the input file path I am opening the corresponding reader and making the FieldSet.

Case I am looking for: I am looking to read these CSV files in single step and make a FieldSet so that in processor I can again split the data into list of TRADES and PORTFOLIO objects. Is there a way I can make the FlatFileItemReader capable of finding what is the CSV picked, and choose the corresponding linemapper..?

Job Definition:

<batch:step id="tradeStep1" allow-start-if-complete="true">
        <batch:tasklet>
            <batch:chunk reader="csvReader"
                processor="csvProcessor" writer="csvWriter"
                commit-interval="1" />
        </batch:tasklet>
        <batch:next on="*" to="tradeStep2" />
        <batch:fail on="FAILED" />
</batch:step>

tradeStep2 will archive processed CSV files.

jeb
  • 78,592
  • 17
  • 171
  • 225
Rajesh Kumar
  • 39
  • 1
  • 11
  • Why not just have 2 separate steps to read the files? Seems like extra work to push them together, just so the processor has to split them back apart. – Dean Clark May 23 '17 at 12:36
  • so that means like step1 will have an empty writer and in step2 I should get place an actual writer. But as I stated I want to know if I can make this in single step..? – Rajesh Kumar May 23 '17 at 12:59
  • You haven't indicated what the writer would do. I assume it would write some output (which would be different for `TRADE` and `PORTFOLIO` objects. If that's the case then I would just have 2 separate Reader/Processor/Writer steps: one for Trades, and one for Portfolios. There are ways to use multiple LineMappers in 1 reader (e.g. `PatternMatchingCompositeLineMapper`, but they don't work that well with CSV) – Dean Clark May 23 '17 at 13:58
  • Yes, writer will try to insert the data into two different tables from the same schema. Currently this is been done with DB2 stored procedures. Here TRADE data will be inserted first, if that goes success PORTFOLIO data will be inserted. But, PORTFOLIO tables primary key will be constructed by referring to TRADES table data. So do you still see it will be better to keep them in separate steps? – Rajesh Kumar May 24 '17 at 05:45
  • Absolutely. Step 1 should process Trades and step 2 should process Portfolios. – Dean Clark May 24 '17 at 13:33

0 Answers0