0

I have a Rails 3.2.20 app which I'm introducing Resque into the environment to background job mail and sms alerts. I have everything setup in development properly and now I'm to the point of preparing to merge my branch and push to staging and then to production for testing. But I have a few questions.

1.) How much memory will I need to run Resque in production (approximately). I understand that by starting up a Resque worker it loads the full environment. I'm a bit tight on memory and don't want to have any issues. I'll be using a single Resque worker as our email/sms traffic is very light and we don't mind queues being backed up for a few seconds to a minute. I know this question is very vague, but I'd like to get a feel for the memory footprint that Resque requires.

2.) I will have Redis running but need to figure out how to start a Resque worker on deployment as well as kill the existing Resque worker. I've come up with the following which I would add as a Cap after deploy action.

task :resque_restart do
  run "kill $(ps aux | grep 'resque' | awk '{print $2}')"
  run "cd #{current_path}; bundle exec rake resque:work QUEUE=*"
end

I haven't actually tested this with Capistrano 2 (which I'm using), but have tested the commands manually and the first command does kill all the resque rake tasks and the second command starts up the worker with all Queues enabled.

I'm not sure if this is the best way to go or not, so I'd really like to hear some feedback on this simple Capistrano task I wrote.

3.) What is the best way to monitor my Resque rake task. So for instance if it crashes or catches a signal to terminate, how can I have it restarted so the app doesn't crash and to assure the worker rake task is always running?

Thanks in advance for any advice or guidance you can provide.

nulltek
  • 3,247
  • 9
  • 44
  • 94

1 Answers1

1
  1. It really depends on the size of your app. From my experience, generally a single Resque worker isn't much larger than your app's footprint. However, if your Resque worker will instantiate a lot of large objects, the size of the Resque instance could grow very quickly.

  2. Check out the capistrano-resque gem. It provides all this functionality for you, plus more. :)

  3. There are several options for this. A lot of people have followed something similar to this post about running Resque in Production and using the God gem. Personally, I've used a process similar to what is described in this post using Monit. Monit can be a bit of a pain to set up, so I'd strongly recommend checking out the God gem.

Community
  • 1
  • 1
CDub
  • 13,146
  • 4
  • 51
  • 68
  • This is good. Really good information. My Resque worker will be used solely for emails and Twilio SMS. It shouldn't instantiate a lot of large objects, just simple email and SMS calls which are ActionMailer and Twilio API calls. I really like the capistrano-resque gem. It's definitely what I'm looking for. I'm looking into God also as I've had a love/hate relationship with Monit for some time now. :) – nulltek Jan 06 '15 at 02:19
  • Yeah in that case I'd definitely expect your resque instance to be extremely small in footprint. Sorry I can't give an exact size, but without knowing your app, it's hard to estimate. – CDub Jan 06 '15 at 02:20
  • No worries, I have a staging environment which I can test this all out on and simulate production traffic. I don't expect Resque to take up more than 200-300MB. I have a 4GB server with about 1.9GB to spare so I should be fine. I'll definitely look into the capistrano-resque gem and using God for process control. If I have any questions, I'll definitely post up. Can't wait for my app to gain the speed of background processing. Actionmailer is pretty fast, but those darn Twilio API calls add 1-2s in latency. – nulltek Jan 06 '15 at 13:03