0

My goal is to upload a file containing rows of firstname and lastname, parse it and create Person model in db for each row.

I do the following and it works fine

file = CSV.parse(the_file_to_parse)
file.each do |row|
  person = Person.new(:firstname => row[0], :lastname => row[1])
  person.save
end

until my file contains accents (french words), I get 

Encoding::UndefinedConversionError: "\xC3" from ASCII-8BIT to UTF-8:
INSERT INTO "people" ("created_at", "firstname", "lastname",
"updated_at") VALUES (?, ?, ?, ?)

What is the best way to deal with this encoding issue?

steenslag
  • 79,051
  • 16
  • 138
  • 171
denisjacquemin
  • 7,414
  • 10
  • 55
  • 72

3 Answers3

2

You need to open the csv file with the right encoding. For example:

require 'csv'
require 'pp'

encoding = "ISO-8859-1"

csv = CSV.open "names.csv", "rb:#{encoding}"
csv.each do |line|
    puts "#{line[0]} #{line[1]}"
end

Here's a hint: it's probably not UTF-8.

The list of encodings that your ruby supports can be viewed with this command in irb:

puts Encoding.list.map(&:to_s).sort
Michael Slade
  • 13,802
  • 2
  • 39
  • 44
0

try to set

# encoding: utf-8

on the top of ruby file. or 

export RUBYOPT=-Ku

before rails s

gayavat
  • 18,910
  • 11
  • 45
  • 55
0

If you have no clue about the encoding of the input file you might try ensure_encoding.

It will try to guess the input encoding and try (does not work for all encodings) to convert it to your preferred target encoding.

maerzbow
  • 185
  • 1
  • 2
  • 9