CSV parse with IFS bash : choose ";" as the separator

Question

I have a CSV with 130 cols and i need to do 3 csv with that. I'm looping with a while and IFS because i need to do someting with the vars on each row.

Here what i did :

while IFS=";" read [my 130 vars]
[what i do with the vars] 
done < file.csv

But i have a problem on some rows because the original csv i receive is like :

"Hi";"i";"got;a problem"

As you can see i have a problem with a ; in a value. And the IFS read it as the separation of two values. So here is my question : is there a way to take ";" as the separator instead of just ; ?

Thank you for your answer, yes i tried it and it didn't work :/ — Neringan, Nov 29 '13 at 11:30

score 3 · Accepted Answer · answered Nov 29 '13 at 12:01

3

You could use awk:

gawk 'BEGIN{FPAT="([^;]+)|(\"[^\"]+\")"}{for(i=1;i<=NF;i++){printf ("%s\n",$i)}}' file.csv

For your input, it'd produce:

"Hi"
"i"
"got;a problem"

(I doubt if it's possible to achieve the desired result using bash, i.e. by manipulating IFS.)

answered Nov 29 '13 at 12:01

devnull

118,548
33
236
227

1

That's it. I could not figure out the awk thing.. +1. – slayedbylucifer Nov 29 '13 at 12:13
@slayedbylucifer I'm not sure what the OP wants but I'd choose Perl over anything else for such tasks. – devnull Nov 29 '13 at 12:16

score 2 · Answer 2 · answered Nov 29 '13 at 11:47

2

if you are OK with perl, then:

# cat version 
"Hi";"i";"got;a problem"

# perl -MText::ParseWords -n -l -e 'print $_ for parse_line(";", 1, $_);' version
"Hi"
"i"
"got;a problem"

I am sure there should be a way to achieve the same with awk

I could manage with sed:

# cat version | sed 's/;\("[^"]*"\)*/\n\1/g'
"Hi"
"i"
"got;a problem"

answered Nov 29 '13 at 11:47

slayedbylucifer

22,878
16
94
123

@devnull in fact yes now i have 15 reputation i can :) – Neringan Nov 29 '13 at 13:21

CSV parse with IFS bash : choose ";" as the separator

2 Answers2

Linked