4

I am desperate to get a streaming CSV download working in my Rails 3.2.2 app.

I have tried the 'csv_builder' gem (https://github.com/dasil003/csv_builder), which advertises support for this feature, however it seems that there were some changes in Rails 3.2 that are keeping it from working (it produces an 'uninitialized constant ActionView::Template::Handler' error on app startup).

Any other ideas or solutions out there? Thanks!

EDIT: To clarify, I am needing to export all entries of a model as a CSV file. There are so many rows, that it is timing out... therefore the need for streaming. I have used the Comma gem (https://github.com/crafterm/comma) for this in the past, but it doesn't support streaming either, at the moment.

Matt Fordham
  • 3,147
  • 10
  • 34
  • 51

2 Answers2

7

OK, after a bit more research I hacked together the following in my controller. It'll stream if response_body is given something enumeratable (is that a word?). Also, the server needs to be able to stream (I am using Unicorn on Heroku). I'd like very much to not have all this stuff in the controller, so my next step is to extract it out somehow. 

  format.csv {
    @entries = Entry.all
    @columns = ["First Name", "Last Name"].to_csv
    @filename = "entries-#{Date.today.to_s(:db)}"

    self.response.headers["Content-Type"] ||= 'text/csv'
    self.response.headers["Content-Disposition"] = "attachment; filename=#{@filename}"
    self.response.headers["Content-Transfer-Encoding"] = "binary"

    self.response_body = Enumerator.new do |y|
      @entries.each_with_index do |entry, i|
        if i == 0
          y << @columns
        end
        y << [entry.first_name, entry.last_name].to_csv
      end
    end
  }
Matt Fordham
  • 3,147
  • 10
  • 34
  • 51
  • Could you provide the full controller definition just to be sure – Jonathan Apr 18 '12 at 22:41
  • This is just killing me trying to get this to work with Rails 3.2.1 using the same code. Is there something to the unicorn config beyond "listen 3000, :tcp_nopush => false"? Anything else that I might be missing beyond the code you've posted above? Any additional insight would be awesome and much appreciated. – Blastula May 31 '12 at 19:05
  • If you've use Rack::ETag, you need to add the following line in order to prevent ETag to buffer the response : self.response.headers["Last-Modified"] = Time.now.to_s – etiennepeiniau Nov 04 '12 at 18:35
  • 1
    Check out this tutorial as well: http://www.jacopretorius.net/2014/02/streaming-data-with-rails-and-heroku.html – John Goodman Oct 09 '14 at 17:44
0

The approach that I took with a Rails 2.3.8 app was to spawn a new thread to handle the csv parsing and then use an AJAX call to check the server to see if the file was ready (I relied on File.mtime).

Just ripped it out of the app to post here so I've removed alot of the csv parsing code, and haven't included all of the views

sorry end of the day rush :D

controllers/exports_controller.rb

    class ExportsController < ApplicationController
            require 'fastercsv'
            require 'generic_agent'
            require 'generic_record'

        def listing

          @this_filepath = "../html/whatever/" << Time.now.strftime("%I:%M:%S_%d:%m:%y") << ".csv"

          @spawn_id = spawn(:nice => 1) do

            FasterCSV.open(@this_filepath, "w") do |csv|

              csv << [ "outbreak_id"]
            end

          end

          render :update do |page|
            page.replace_html 'export_status', :partial => 'export_status_partial'
          end

        end

  def send_export

    @this_filepath = params[:with]
    csv_file = File.open(@this_filepath.to_s, 'r')

    csv_string = ""
    csv_file.each_line do |line|
      csv_string << line
    end

   send_data csv_string, :filename => "export.csv",
                :type => 'text/csv; charset=iso-8859-1; header=present',
                :disposition => "attachment; filename=export.csv"
                #send_file @this_filepath.to_s, :stream => false, :type=>"text/csv", :x_sendfile=>true

                #send_data csv_string, :filename => export.csv

                #File.delete(@this_filepath.to_s)
  end

  def export_checker
    filename_array = params['filename'].split(/\//)
                @file_found = 0
                @file_ready = 0

                @file_size = File.size(params['filename'])
                @this_filepath = params['filename']

                if File.exists?(params['filename'])
                release_time = Time.now - 5.seconds
                if File.mtime(params['filename']).utc < release_time.utc

                @file_found = 1
                @file_ready = 1
                @file_access_time = File.mtime(params['filename'])
                @file_release_time = release_time
                @file_size = File.size(params['filename'])

                else
                @file_found = 1
                @file_ready = 0
                @file_size = File.size(params['filename'])

                end

                else

                @file_found = 0
                @file_ready = 0
                @file_size = File.size(params['filename'])

                end

    render :action => "export_checker"
  end
end

views/exports/export_checker.rjs

if @file_found == 1 && @file_ready == 1 && @file_size > 0


page.replace_html 'link_to_file', :partial => "export_ready"
if @file_release_time
page.replace_html 'notice', "<div>Completed #{@file_release_time.strftime("%I:%M:%S %A %d %B %Y")} :: file size #{@file_size.to_s}</div>"
end

page.visual_effect :highlight, 'link_to_file', :endcolor => '#D3EDAB'

elsif @file_found == 1
page.replace_html 'link_to_file', "<div> File found, but still being constructed.</div><div>#{@this_filepath.to_s}</div>"
page.visual_effect :highlight, 'link_to_file', :endcolor => '#FF9900'
else
page.replace_html 'link_to_file', "<div> File not found @file_found #{@file_found.to_s} @file_ready #{@file_ready.to_s}</div>"
page.visual_effect :highlight, 'link_to_file', :endcolor => '#FF0000'
end
Pasted
  • 864
  • 1
  • 12
  • 22