failed to allocate memory

Question

I have a large csv file which is 12 mb in size. I do have a 12 Gig of RAM in my laptop but I am still running out of memory. I have rebooted my laptop and started the program fresh but I still have issues. I am using SmarterCSV hoping that it will manage large csv files but no luck so far.

    C:\>irb
irb(main):001:0> require 'smarter_csv'
=> true
irb(main):002:0> filename="C:\\Users\\Dino\\Downloads\\mktsymbols_v2.txt"
=> "C:\\Users\\Dino\\Downloads\\mktsymbols_v2.txt"
irb(main):003:0> rows=SmarterCSV.process(filename)
NoMemoryError: failed to allocate memory
        from C:/Ruby23/lib/ruby/2.3.0/irb/inspector.rb:109:in `inspect'
        from C:/Ruby23/lib/ruby/2.3.0/irb/inspector.rb:109:in `block in <module:IRB>'
        from C:/Ruby23/lib/ruby/2.3.0/irb/inspector.rb:102:in `inspect_value'
        from C:/Ruby23/lib/ruby/2.3.0/irb/context.rb:384:in `inspect_last_value'
        from C:/Ruby23/lib/ruby/2.3.0/irb.rb:661:in `output_value'
        from C:/Ruby23/lib/ruby/2.3.0/irb.rb:490:in `block (2 levels) in eval_input'
        from C:/Ruby23/lib/ruby/2.3.0/irb.rb:623:in `signal_status'
        from C:/Ruby23/lib/ruby/2.3.0/irb.rb:486:in `block in eval_input'
        from C:/Ruby23/lib/ruby/2.3.0/irb/ruby-lex.rb:246:in `block (2 levels) in each_top_level_statement'
        from C:/Ruby23/lib/ruby/2.3.0/irb/ruby-lex.rb:232:in `loop'
        from C:/Ruby23/lib/ruby/2.3.0/irb/ruby-lex.rb:232:in `block in each_top_level_statement'
        from C:/Ruby23/lib/ruby/2.3.0/irb/ruby-lex.rb:231:in `catch'
        from C:/Ruby23/lib/ruby/2.3.0/irb/ruby-lex.rb:231:in `each_top_level_statement'
        from C:/Ruby23/lib/ruby/2.3.0/irb.rb:485:in `eval_input'
        from C:/Ruby23/lib/ruby/2.3.0/irb.rb:395:in `block in start'
        from C:/Ruby23/lib/ruby/2.3.0/irb.rb:394:in `catch'
        from C:/Ruby23/lib/ruby/2.3.0/irb.rb:394:in `start'
        from C:/Ruby23/bin/irb.cmd:19:in `<main>'
Maybe IRB bug!

I don't really think it's an irb bug. Does it happen when you're running the script file with ruby? — Jakub Groncki, Aug 03 '17 at 12:11
Then it's probably the library error. You can file a bug report. Last question - what happens when you try to process the file with the chunk_size option set? — Jakub Groncki, Aug 03 '17 at 18:46
Something along the lines of: rows = SmarterCSV.process(filename, :chunk_size => 2) or chunk_size: 2, if you don't like hash rockets. — Jakub Groncki, Aug 05 '17 at 06:45

failed to allocate memory

0 Answers0