Tab delimited file parsing in Rails

Question

I have a file that looks like this

ID Name  Car
1  Mike  Honda
2  Adam  Jim

These values are tab delimited, and from this I want to parse it in Ruby and put it into my database.

I have tried the following

require 'csv'

CSV.foreach("public/files/example.tab", {:col_sep => "\t"}) do |row|
  @stuff = row[0]
end

but @stuff just returns the whole entire object, and doesn't seem to be using the column separator I specified.

It also does not take into account that the first row is a header.

How can I parse a tab delimited file in Ruby and how do I tell it that the first row is a header?

Thank you, coincidentally :col_sep => "\t" saved my day :D – Romans 8.38-39 Jul 02 '14 at 10:03 — Romans 8.38-39, Jul 02 '14 at 10:03

user1047725 · Accepted Answer · 2011-11-18T08:21:19.977

4

I have had success with FasterCSV and Ruby 1.8.7, I believe it's now the core csv library in 1.9, using this:

table = FasterCSV.read(result_file.to_file.path, { :headers => true, :col_sep => "\t", :skip_blanks => true })
unless table.empty?
    header_arry = Array.new
    table.headers.each do |h|
      #your header logic, e.g.
      # if h.downcase.include? 'pos'
        # header_arry << 'position'
      # end
      # simplest case here
      header_arry << h.downcase
      #which produces an array of column names called header_arry
    end

    rows = table.to_a
    rows.delete_at(0)
    rows.each do |row|
      #convert to hash using the column names
      hash = Hash[header_arry.zip(row)]
      # do something with the row hash
    end
  end

edited Nov 18 '11 at 08:21

answered Nov 17 '11 at 23:01

user1047725

84
4

"undefined local variable or method `header_arry'" - The text file doesn't have the header defined in any special way I don't think... Sorry I'm very new to rails – Mike Silvis Nov 18 '11 at 00:39
I have added a bit more explanation in the header section. AFAIK it just uses the first row, nothing needs defining as haeder – user1047725 Nov 18 '11 at 08:25

Tilo · Answer 2 · 2012-10-15T21:02:15.097

Update

Check out the Gem "smarter_csv" https://github.com/tilo/smarter_csv/ ; it has a couple of interesting features to create hashes from CSV data.

Previous Answer

here's how I'd do it (along the way I convert the "arrays of arrays" which are returned by CSV.read or CSV.parse, into "arrays of hashes"... this makes the data look more like ActiveRecord data, and it's a bit easier to process this way later on..

require 'csv'

def process(csv_array)  # makes arrays of hashes out of CSV's arrays of arrays
  result = []
  return result if csv_array.nil? || csv_array.empty?
  headerA = csv_array.shift             # remove first array with headers from array returned by CSV
  headerA.map!{|x| x.downcase.to_sym }  # make symbols out of the CSV headers
  csv_array.each do |row|               #    convert each data row into a hash, given the CSV headers
    result << Hash[ headerA.zip(row) ]  #    you could use HashWithIndifferentAccess here instead of Hash
  end
  return result
end

# reading in the CSV data is now just one line:

csv_data = process( CSV.read( filename , { :col_sep => "\t"}) )

 => [{:id=>"1", :name=>"Mike", :car=>"Honda"}, 
     {:id=>"2", :name=>"Adam", :car=>"Jim"}]

you can now process the data like this:

csv_data.each do |hash|
  # ...
end

Tab delimited file parsing in Rails

3 Answers3

Update

Previous Answer

See also: