Read large amount of data from CSV and XML file Using Apache camel

Question

Is it possible to read large amount of data CSV and XML files(approximately more than 1GB each file) using Apache Camel?

If any performance issues or limitations involved in this case, what kind of solutions are available from Apache Camel.

Answer 1

Answer about csv is here answer

For consuming large xml file you can use http://camel.apache.org/stax.html

Answer 2

After analyzing the issue,I found a following Solution.

So 1 GB is not soooo much,if we can simply spend enough RAM on the Camel.

It depends on the following question

Do we need the 1 GB at the same time to access or Do we access all the files in any order?

If not, then we would have to "stream" the CSV / XML file, so read as InputStream and then sequentially get what we need.

CSV:

<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
    <from uri="direct:start"/>
    <split streaming="true">
        <tokenize token="\n"/>
        <to uri="mock:result"/>
    </split>
     <unmarshal><csv /></unmarshal>
</route>

XML:

 <camelContext xmlns="http://camel.apache.org/schema/spring">
  <route>    
    <from uri="direct:start"/>
    <split streaming="true">     
       <ref>staxRecord</ref>     
      <to uri="mock:result"/>
    </split>
  </route>
</camelContext>