How to read and process a file in Ruby with EventMachine

Question

I am wondering if anybody figured out a way to process bigger (500Mb+) text files with EventMachine, where you actually need to access the individual lines.

Answer 1

I guess I found the answer, the only thing is messy is the read_chunk gets called in the after io.gets, and I am not sure why it works :)

require 'eventmachine'
def process_line(line)
  puts line
end
EM.run do
  io = File.open('query_profiles.csv')
  read_chunk = proc do
    if line = io.gets
      process_line(line)
      EM.next_tick(read_chunk)
    else
      EM.stop
    end
  end
  EM.next_tick(read_chunk)
end