How to prevent read() method from splitting based on delimiter inside quotation marks in csv?

Asked Aug 15 '19 at 16:09

Active Aug 16 '19 at 09:03

Viewed 76 times

For example, if I have something like

"I like, cookies"

I do NOT want Spark's read() to csv method to split between the like and cookies. I want it to be parsed as "I like, cookies". I thought this was common csv practice, but this is being separated.

I saw similar questions mentioned using Databricks but I am trying to avoid that. Is there any other way to prevent this from happening?

edited Aug 16 '19 at 09:03

Bartosz Konieczny

1,985
12
27

asked Aug 15 '19 at 16:09

aberlasters

The delimiter is a comma so I do not want a delimiter inside quotation marks to separate. – aberlasters Aug 15 '19 at 18:20
Is there any specific reason you are using csv reader ? – Amit Aug 15 '19 at 19:02
I get that. Can u show what options have u tried so far and also with some more sample data if you have. – vikrant rana Aug 23 '19 at 21:17

How to prevent read() method from splitting based on delimiter inside quotation marks in csv?

0 Answers0