Ruby on Rails: How to run things in the background?

Question

When a new resource is created and it needs to do some lengthy processing before the resource is ready, how do I send that processing away into the background where it won't hold up the current request or other traffic to my web-app?

in my model:

class User < ActiveRecord::Base
 after_save :background_check

 protected
 def background_check
  # check through a list of 10000000000001 mil different
  # databases that takes approx one hour :)
  if( check_for_record_in_www( self.username ) )
    # code that is run after the 1 hour process is finished.
    user.update_attribute( :has_record )
  end
 end
end

score 42 · Accepted Answer · edited Oct 04 '12 at 08:04

42

You should definitely check out the following Railscasts:

They explain how to run background processes in Rails in every possible way (with or without a queue ...)

edited Oct 04 '12 at 08:04

AnkitG

6,438
7
44
72

answered May 22 '09 at 14:38

ujh

4,023
3
27
31

2

These days, the most-used/best-supported options are [sidekiq](https://github.com/mperham/sidekiq), [resque](https://github.com/defunkt/resque), and [delayed_job](https://github.com/collectiveidea/delayed_job). IMHO, Sidekiq is the best one for high-throughput queues, and delayed_job is the best one for very low throughput. – John Douthat Jan 29 '13 at 03:02
Another option is [IronWorker](http://www.iron.io) if you want background processing "as a service" and never have to worry about the idea of servers or queuing. IronMQ/IronWorker also offers auto retries, task logs, webhooks, etc. It integrates nicely with Rails. Here are some videos: https://www.youtube.com/user/ironiodevelopers – Chad May 01 '14 at 22:27
Useful links and still applicable after these years but it doesn't summarize them or answer the question directly. – Peter DeWeese Dec 15 '18 at 14:54

score 8 · Answer 2 · edited Apr 14 '11 at 17:46

I've just been experimenting with the 'delayed_job' gem because it works with the Heroku hosting platform and it was ridiculously easy to setup!!

Add gem to Gemfile, bundle install, rails g delayed_job, rake db:migrate Then start a queue handler with;

RAILS_ENV=production script/delayed_job start

Where you have a method call which is your lengthy process i.e

company.send_mail_to_all_users

you change it to;

company.delay.send_mail_to_all_users

Check the full docs on github: https://github.com/collectiveidea/delayed_job

score 7 · Answer 3 · answered May 22 '09 at 12:44

Start a separate process, which is probably most easily done with system, prepending a 'nohup' and appending an '&' to the end of the command you pass it. (Make sure the command is just one string argument, not a list of arguments.)

There are several reasons you want to do it this way, rather than, say, trying to use threads:

Ruby's threads can be a bit tricky when it comes to doing I/O; you have to take care that some things you do don't cause the entire process to block.
If you run a program with a different name, it's easily identifiable in 'ps', so you don't accidently think it's a FastCGI back-end gone wild or something, and kill it.

Really, the process you start should be "deamonized," see the Daemonize class for help.

score 2 · Answer 4 · answered May 22 '09 at 12:57

you ideally want to use an existing background job server, rather than writing your own. these will typically let you submit a job and give it a unique key; you can then use the key to periodically query the jobserver for the status of your job without blocking your webapp. here is a nice roundup of the various options out there.

score 1 · Answer 5 · answered Mar 29 '10 at 14:47

1

I think spawn is a great way to fork your process, do some processing in background, and show user just some confirmation that this processing was started.

answered Mar 29 '10 at 14:47

Vojto

6,901
4
27
33

score 1 · Answer 6 · answered May 22 '09 at 16:26

1

I like to use backgroundrb, its nice it allows you to communicate to it during long processes. So you can have status updates in your rails app

answered May 22 '09 at 16:26

Matthew Campbell

401
1
4
7

Edu · Answer 7 · 2009-05-22T13:06:04.707

0

What about:

def background_check
   exec("script/runner check_for_record_in_www.rb #{self.username}") if fork == nil
end

The program "check_for_record_in_www.rb" will then run in another process and will have access to ActiveRecord, being able to access the database.

edited May 22 '09 at 13:06

answered May 22 '09 at 12:58

Edu

1,949
1
16
18

Ruby on Rails: How to run things in the background?

7 Answers7

Linked