error while reading csv file from remote url

Question

I am reading CSV file from the remote location and then parsing it to store into the local collection, however, I am getting following error and the response is getting as some garbage characters

Encoding::UndefinedConversionError: "\xD0" from ASCII-8BIT to UTF-8

Here is code to fetch the csv from the remote location

    uri = URI.parse(site_url)
    data = {'token' => "xxxxxxxxxxxxxx"}
    req = Net::HTTP::Get.new(uri.path)
    req.set_form_data(data)
    http = Net::HTTP.new(uri.host, uri.port)
    if uri.scheme =='https'
      pem = File.read("#{Rails.root}/config/ca-certificates.crt")
      http.use_ssl = true
      http.cert = OpenSSL::X509::Certificate.new(pem)
      http.verify_mode = OpenSSL::SSL::VERIFY_PEER
    end
    response = http.start do |httpvar|
      httpvar.request(req)
    end
    if response.kind_of?(Net::HTTPRedirection)
      headers = {'Content-Type' => "text/csv; ecoding=utf-8" }
      uri = URI.parse(response['location'])
      response = open(uri).read
    end
     CSV.parse(response, headers: true, header_converters: :symbol) do |row|
      puts row
    end

How can I reproduce your problem? Can you write a [MCVE]? What is `response`? Which line does the error message originate from? — Tom Lord, Apr 10 '18 at 13:30
You've mis-spelt `ecoding` (should be `encoding`), but that's doesn't seem to be relevant since you're not actually using this variable anywhere? — Tom Lord, Apr 10 '18 at 13:32

score 0 · Answer 1 · answered Apr 10 '18 at 14:15

0

From Encoding error while writing data from excelfile to database (mysql)

Try adding

# ecoding: utf-8

To your .rb file, or alternatively use the magic_encoding gem

answered Apr 10 '18 at 14:15

Mark

6,112
4
21
46

I'm somewhat baffled how the OP, *and* that other answer, *and* you have all mis-spelt "e**n**coding" the same way! – Tom Lord Apr 11 '18 at 10:01

score 0 · Answer 2 · answered Apr 10 '18 at 14:34

I assume this error is occurring when you call CSV.parse on the response. You can force the proper encoding on the response string before parsing it, like this:

response.force_encoding(Encoding::UTF_8)
CSV.parse(response, headers: true, header_converters: :symbol) do |row|
  puts row
end

error while reading csv file from remote url

2 Answers2