Ruby csv error : invalid byte sequence in UTF-8

Question

[sagar@BL-53 RcTools]$ irb
1.9.3p0 :001 > require 'csv'
 => true
1.9.3p0 :002 > master = CSV.read("./public/jobs/in/Appexchange_Applications_Companies_487.csv")
ArgumentError: invalid byte sequence in UTF-8
        from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1855:in `sub!'
        from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1855:in `block in shift'
        from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1849:in `loop'
        from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1849:in `shift'
        from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1791:in `each'
        from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1805:in `to_a'
        from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1805:in `read'
        from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1411:in `block in read'
        from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1354:in `open'
        from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1411:in `read'
        from (irb):2
        from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/bin/irb:16:in `<main>'
1.9.3p0 :003 >

But when i do

1.9.3p0 :003 > master = CSV.open("./public/jobs/in/Appexchange_Applications_Companies_487.csv","r")
 => <#CSV io_type:File io_path:"./public/jobs/in/Appexchange_Applications_Companies_487.csv" encoding:UTF-8 lineno:0 col_sep:"," row_sep:"\r\n" quote_char:"\"">
1.9.3p0 :004 >

I just want to know why this is happening and what is solution. And i want to read the csv because it returns an array of that csv. So if i read file in first way like

master = CSV.read("./public/jobs/in/Appexchange_Applications_Companies_487.csv")

It returns me an array

1.9.3p0 :008 > master.class
 => Array

But in second case, class is CSV. What is solution to read csv in first way.

Is the csv file in UTF-8? If not, you can specify the encoding by passing it as a second argument to CSV.read. — jlundqvist, Aug 14 '12 at 12:53
ruby iconv can handle encoding issue . please have look http://stackoverflow.com/questions/1793284/uploaded-file-char-set-conversion-with-ruby — Chetan Muneshwar, Aug 14 '12 at 13:56

score 0 · Answer 1 · answered Aug 14 '12 at 15:23

Regarding the error: First, make sure that you are using the correct character encodings. If you do, then you have probably invalid data in your csv file. You can probably fix it using iconv (see link posted by Chetan Muneshwar).

Regarding the second part of your question: CSV.open just opens the file for reading but does no reading yet. CSV.read will open the file, read its contents, and close it again. So to just get the data out of the file use CSV.read.

Ruby csv error : invalid byte sequence in UTF-8

1 Answers1