7

For some reason processing images (carrierwave + minimagick) stops working about a week after the service is started. There is no unusual increase in traffic or upload processes. Once the ENOMEM error occurs everything seems to 'lock up' and any following processes fail as well.

How can I 'rescue' the system once it is in that behavior or even prevent this from happening?

Some of the errors:

Errno::ENOMEM (Cannot allocate memory - export LANG=C && identify -ping /tmp/mini_magick20111219-18047-1dhmawm.jpg 2>&1):
   app/uploaders/photo_uploader.rb:70:in `custom_thumbnail'
   app/controllers/upload_controller.rb:186:in `process_upload'
   app/middleware/flash_session_cookie_middleware.rb:17:in `call'
   app/middleware/flash_session_cookie_middleware.rb:17:in `call'


Errno::ENOMEM (Cannot allocate memory - export LANG=C && mogrify -format jpg /tmp/mini_magick20111219-18047-1c43qpf.jpg 2>&1):
   app/controllers/upload_controller.rb:186:in `process_upload'
   app/middleware/flash_session_cookie_middleware.rb:17:in `call'
   app/middleware/flash_session_cookie_middleware.rb:17:in `call'

...
...

Errno::ENOMEM (Cannot allocate memory - export LANG=C && mogrify -resize "120x180" -gravity "Center" -extent "120x120" /tmp/mini_magick20111219-18047-155ofje.jpg 2>&1):
   app/controllers/upload_controller.rb:186:in `process_upload'
   app/middleware/flash_session_cookie_middleware.rb:17:in `call'
   app/middleware/flash_session_cookie_middleware.rb:17:in `call'

Memory and swap space info:

Mem:   8193476k total,  7907152k used,   286324k free,     5968k buffers
Swap: 12396808k total,  9494924k used,  2901884k free,   180308k cached

Version information:

  • Rails 3.0.6
  • ruby 1.9.2p290
  • Apache 2.2.14
  • passenger 3.0.9

Thank you!

lulalala
  • 17,572
  • 15
  • 110
  • 169
Florian
  • 373
  • 5
  • 18
  • 1
    A bunch of memory leaks have been fixed in ruby 1.9.3. I'd suggest upgrading and see if that helps. http://svn.ruby-lang.org/repos/ruby/tags/v1_9_3_0/ChangeLog – Thilo Dec 19 '11 at 19:57
  • Thank you Thilo! Since the update memory is being handled a lot better. – Florian Dec 21 '11 at 17:20

2 Answers2

1

I can't find the proper fix, but once I restarted using capistrano (cap deploy:restart) things starts to run again.

I use Ruby 1.9.3p194 (2012-04-20 revision 35410)

The problem starts when I use mini_magick in Carrierwave to manipulate!

  def strip
    manipulate! do |img| #where problem occurs
      img.strip
      img = yield(img) if block_given?
      img
    end
  end

My log:

{Cannot allocate memory - identify -ping /tmp/mini_magick20121019-3337-pg50p9.jpg shared/bundle/ruby/1.9.1/gems/subexec-0.2.1/lib/subexec.rb:71:in spawn' shared/bundle/ruby/1.9.1/gems/subexec-0.2.1/lib/subexec.rb:71:in spawn' shared/bundle/ruby/1.9.1/gems/subexec-0.2.1/lib/subexec.rb:55:in run!' shared/bundle/ruby/1.9.1/gems/subexec-0.2.1/lib/subexec.rb:41:inrun' shared/bundle/ruby/1.9.1/gems/mini_magick-3.4/lib/mini_magick.rb:405:in run' shared/bundle/ruby/1.9.1/gems/mini_magick-3.4/lib/mini_magick.rb:399:in run_command' shared/bundle/ruby/1.9.1/gems/mini_magick-3.4/lib/mini_magick.rb:198:in valid?' shared/bundle/ruby/1.9.1/gems/mini_magick-3.4/lib/mini_magick.rb:163:in create' shared/bundle/ruby/1.9.1/gems/mini_magick-3.4/lib/mini_magick.rb:73:in read' shared/bundle/ruby/1.9.1/gems/mini_magick-3.4/lib/mini_magick.rb:134:in block in open' shared/bundle/ruby/1.9.1/gems/mini_magick-3.4/lib/mini_magick.rb:133:in open' shared/bundle/ruby/1.9.1/gems/mini_magick-3.4/lib/mini_magick.rb:133:in open' shared/bundle/ruby/1.9.1/gems/carrierwave-0.6.2/lib/carrierwave/processing/mini_magick.rb:245:in `manipulate!'

lulalala
  • 17,572
  • 15
  • 110
  • 169
1

I believe this is the same issue as documented here. Different gems, but same effect: http://adamniedzielski.github.io/blog/2014/02/05/fighting-paperclip-errno-enomem-error/

The upshot of this article is that "To create the child process, free memory must be greater than the memory taken by the parent process.". This recommends the use of the posix-spawn to avoid the issue. This doesn't appear to be provided in Minimagick, so some kind of patch or workaround would be required.

jonathannen
  • 91
  • 1
  • 3