Spring Batch: DelimitedLineTokenizer with line-breaks (newline) in CSV

Asked Sep 18 '14 at 14:06

Active Sep 18 '14 at 16:33

Viewed 1,717 times

I need to parse an semicolon-separated CSV-File with Spring Batch. Currently I'm using DelimitedLineTokenizer within a DefaultLineMapper.

The Problem ist, that my CSV rows contains line breaks between quotes. Its kinda multi-line description of every row.

Example:

column1;column2;column3;column4
value1;"value2
continuation of value2";column3;column4

This leads into the following error:

org.springframework.batch.item.file.transform.IncorrectTokenCountException: Incorrect number of tokens found in record: expected 4 actual 3

So how to tell that Tokenizer to ignore line-breaks \n within the quote-escaped columns?

edited Sep 18 '14 at 16:33

asked Sep 18 '14 at 14:06

Micronax

Your sample isn't quoted…is it in reality? – Michael Minella Sep 18 '14 at 15:02
Sorry, added quotation.. This is just a mock, the original file has about 100 columns and several thousand rows.. – Micronax Sep 18 '14 at 15:15
Just to be clear, I'm assuming it's a semicolon after value1 as well? – Michael Minella Sep 18 '14 at 15:27
2

I just added a test to confirm that scenario works and it does. Are you using a custom `RecordSeperatorPolicy`? Your configuration would be helpful... – Michael Minella Sep 18 '14 at 15:39
Argh. The result of endless copy+paste. – Micronax Sep 18 '14 at 16:33
@Micronax did you get a solution? – saurabh gupta Sep 08 '20 at 09:41
Try this: https://gist.github.com/micronax/3e3d41158c58112d5b4df26df110c063 – Micronax Sep 08 '20 at 13:27

0 Answers0