Using fork in Ruby on Rails for creating parallel process

Question

I have a Rails 3 app in production with Passenger on Apache. I have this code:

class Billing < ActiveRecord::Base
  after_save :sendEmails

  private
    def sendEmails
      fork do 
        UserMailer.clientBilling(self.user, self).deliver
      end
    end
end

In localhost, when the app creates a billing, after it is saved, the app sends an email to the user, everything works fine. But in the server, after the app creates a billing, it throws me errors related to the gem MySQL2, errors like "MySQL server has gone away" or "Connection lost", and the app doesn't send the emails. If I remove the fork it works fine, but I want to use fork, I want to create a separated process because it takes to long when sending emails. What could be the problem?

score 19 · Accepted Answer · edited May 18 '21 at 15:55

19

The problem is that a forked process inherits some of its parent's resources, such as its file descriptors. In particular one such shared resource is the MySQL connection. When the child process finishes its email sending and exits it closes the MySQL connection, which closes the parent processes connection.

If you do continue down this path (and it is frought with similar subtleties) then you need to do something like this:

# Clear existing connections before forking to ensure they do not get inherited.
::ActiveRecord::Base.clear_all_connections! 

fork do
  # Establish a new connection for each fork.
  ::ActiveRecord::Base.establish_connection 
  
  # The rest of the code for each fork...
end

You'll have to do similar thing with services like memcached or mongodb if you use those.

edited May 18 '21 at 15:55

Joshua Pinter

45,245
23
243
245

answered Jan 18 '12 at 18:28

Frederick Cheung

83,189
8
152
174

do you know which resources are inherited? i thought a process fork copies the whole process? isn't the whole rails environment loaded again on a fork? – Andre Schweighofer Jan 22 '12 at 14:10
1

It copies the entire process, but file descriptors refer to the exact same file (see the man page for fork) – Frederick Cheung Jan 22 '12 at 15:26
1

I've found that `establish_connection` isn't always necessary as ActiveRecord will usually manage this transparently. – spume Aug 31 '16 at 14:46
1

I think active record does this automatically now, but I think that it is a relatively recent change – Frederick Cheung Aug 31 '16 at 14:47
@FrederickCheung You're an absolute genius. And well done explaining it, not just providing a solution. This explanation makes a lot of sense. Even (especially?) in 2021. Thanks! – Joshua Pinter May 18 '21 at 15:44

score 9 · Answer 2 · answered Jan 18 '12 at 16:20

9

Be extremely careful when using fork with rails/passenger, it can become very messy! Instead, you should use resque or delayed_job for this task!

answered Jan 18 '12 at 16:20

J-_-L

9,079
2
40
37

2

I took 7 days to discover that the error msg "Mysql::Error: Lost connection to MySQL server during query" and "Mysql::Error: MySQL server has gone away" was a result of the use of fork. – PH. Nov 02 '13 at 00:58

score 3 · Answer 3 · answered Mar 21 '12 at 08:24

3

You can reestablish the connection inside of the fork:

dbconfig = YAML::load(File.open('your_app_dir/config/database.yml'))
ActiveRecord::Base.establish_connection(dbconfig['development'])

answered Mar 21 '12 at 08:24

pavlovt

51
1
2

Using fork in Ruby on Rails for creating parallel process

3 Answers3

Linked