How can I read CSV with no quote_char in ruby?

Question

I have a TSV file with no quote chars. Whenever a \t occurs in the data, it is always to separate columns, and never a part of a column value. Whenever a " occurs, it is always a part of a column value, and never to enclose column values.

I would like to read this CSV in Ruby but it gives me

/Users/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/csv.rb:1925:in `block (2 levels) in shift': Illegal quoting in line 9506. (CSV::MalformedCSVError)

My code is:

CSV.foreach(input_file, { :col_sep => "\t", :headers => true}) do |row|
   puts row
end

Any way to get around this problem?

I highly recommend the fastercsv gem by James Edward Gray II. It makes life simpler. — 6ft Dan, Apr 30 '14 at 00:14
possible duplicate of [Importing CSV quoting error is driving me nuts](http://stackoverflow.com/questions/8073920/importing-csv-quoting-error-is-driving-me-nuts) — mu is too short, Apr 30 '14 at 01:40
@6ftDan: The standard library csv in Ruby 1.9.3 *is* fastercsv AFAIK. — mu is too short, Apr 30 '14 at 01:40

score 31 · Answer 1 · answered Apr 30 '14 at 00:28

31

Turns out I could fix it by putting quote_char => "\x00" to trick it into thinking the zero byte is the quote char.

answered Apr 30 '14 at 00:28

Popcorn

5,188
12
54
87

score 3 · Answer 2 · answered Jan 05 '18 at 14:24

The liberal_parsing option is available for cases like this. From the documentation:

When set to a true value, CSV will attempt to parse input not conformant with RFC 4180, such as double quotes in unquoted fields.

In your example this would be:

CSV.foreach(input_file, { :col_sep => "\t", :headers => true, :liberal_parsing => true }) do |row|
  puts row
end

How can I read CSV with no quote_char in ruby?

2 Answers2