6

Is this going to cause memory issues with Ruby. I know Open-URI writes to a TempFile if the size goes over 10KB. But will HTTParty try and save the whole PDF to memory before it writes to TempFile?

src = Tempfile.new("file.pdf")
src.binmode
src.write HTTParty.get("large_file.pdf").parsed_response
maletor
  • 7,072
  • 7
  • 42
  • 63

1 Answers1

12

You can use Net::HTTP. See the documentation (in particular the section titled "Streaming Response Bodies").

Here's the example from the docs:

uri = URI('http://example.com/large_file')

Net::HTTP.start(uri.host, uri.port) do |http|
  request = Net::HTTP::Get.new uri.request_uri

  http.request request do |response|
    open 'large_file', 'w' do |io|
      response.read_body do |chunk|
        io.write chunk
      end
    end
  end
end
Mark Thomas
  • 37,131
  • 11
  • 74
  • 101
  • 1
    Awesome! Somewhat related question but how are you doing this for your large file uploads? – maletor Feb 16 '12 at 03:49
  • 4
    It's also might be better if you open file for writing in binary mode, simply add `b` flag, like: `open(filename, 'wb') { |io| ... }`. – Dan K.K. Feb 20 '14 at 23:27
  • Didn't debug it deeply but seems like it downloads "in background" while program runs further. – Nakilon Sep 19 '16 at 13:09