Having trouble seeding CSV file into Rails App

Question

I'm trying to seed about 2K records into a model and I've tried using all the different methods out there so far (faster csv, fast_seeder, and the railscasts ep). I feel like this should be pretty simple. I've got all the data in a CSV file and I've got the model set up already, ready to be seeded.

The only thing that's worked so far is what's shown in the RailsCasts episode. I plugged in this code for my attributes, and it seeded only the first line (out of 2K), and that's it:

Country.delete_all
open("#{Rails.root}/path_to_my_file") do |models|
  models.read.each_line do |model|
     column_1, column_2, column_3, column_4 = model.chomp.split(",")
     Model.create!(:attr_1 => column_1, :attr_2 => column_2, ...etc)
  end
end

Then I tried using FasterCSV based on some of the other questions, but I received a bunch of errors saying that fastercsv was already included in the latest release of ruby, I couldn't seem to figure it out (this could be my fault, but I haven't been able to find a good SO question that lays it out nicely).

Finally, Fast_Seeder seemed to have some potential, and it recognized all my entries, but didn't save any of them into the model because I received this error:

SQLite3::SQLException: too many terms in compound SELECT:(all my columns)

Anyway, again I feel like this should be simple enough. I just have a CSV with 2K entries, and a blank model that I need to seed. Best way to do this would be much appreciated, thanks!

jbearden · Accepted Answer · 2013-11-11T05:48:55.300

14

Best success I've seen with this is by doing a rake task.

require 'csv'

namespace :csv do

  desc "Import CSV Data"
  task :import_stuff => :environment do

    csv_file_path = 'db/data.csv'

    CSV.foreach(csv_file_path) do |row|
      Model.create!({
        :column1 => row[0],
        :column2 => row[1],
        :column3 => row[2],        
      })
      puts "Row added!"
    end
  end
end

Put this in your lib/tasks folder with a .rake extension and to run it type: "rake csv:import_stuff"

Also, you might be reaching some limitations with SQL lite... I would suggest checking out MongoDB. It seems like it would be fitting for your current situation. Good luck!

edited Nov 11 '13 at 05:48

answered Aug 20 '12 at 09:02

jbearden

1,869
2
19
23

Thank you! Worked for me. Wondering if the `row =` is necessary. I left it out and worked fine. – BenU Nov 11 '13 at 01:36
Definitely not necessary. Good catch I removed it from the example :) – jbearden Nov 11 '13 at 05:53

score 4 · Answer 2 · answered Aug 19 '12 at 17:32

Looks like I figured it out, sorry about that. I would appreciate some comments on this code, if it needs to be improved.

Anyway, I think I figured it out using fasterCSV (or just CSV). This seems to have worked:

require 'csv'

Model.delete_all
CSV.foreach("#{Rails.root}/lib/data/model.csv") do |row|
     Model.create!(:attr_1 => row[0], :attr_2 => row[1], :attr_3 => row[2], etc)
end

Having trouble seeding CSV file into Rails App

2 Answers2