Rails 5 starting thread from initializer gone crazy

Question

I have the following code in config/application.rb:

config.after_initialize do
  MyConcern.init_processing
end

Then the following related methods:

def init_processing
  @queue_processor = Thread.new do
    process_op_queue
  end
  @queue_processor.name = "Some Queue"
  # @queue_processor_notifier = Queue.new
end

def process_op_queue
  while true
    req = MyRequest.where(done: Helper::FALSE_NIL_OR_MISSING, started_at: nil).order_by(id: :asc).find_one_and_update({'$currentDate' => {started_at: true}}, return_document: :after)
    req ? process_queue_item(req) : sleep(30)
  end
end

Basically starting a thread on server startup to perform some background tasks that are accepted by the app from users by HTTP calls.

In development mode everything works fine. Running in production env with same puma server, the @queue_processor shows dead (from rails console):

MyConcern.instance_variable_get("@queue_processor").join
=> #<Thread:0x00000002b8fa68@MyConcern Queue@/var/lib/openshift/5842d71a5110e25cdf00000c/app-root/runtime/repo/app/controllers/concerns/my_concern.rb:23 dead> 
MyConcern.instance_variable_get("@queue_processor").value
=> false

First it is (mildly said) non-obvious to me how a thread can die without an exception and returning false with the loop that I have above in process_op_queue. The other very interesting facts are that:

there is actually a thread doing the work as obvious to me while doing requests to the app and looking at what changes in the database
in rails console I see only one thread:

.

2.3.1 :009 > Thread.list
 => [#<Thread:0x000000012f63a8 run>] 
2.3.1 :010 >

Part of the app is checking the queue processor thread and reporting errors with it so showing up as dead is really breaking that logic. Any advises how to debug and resolve the issue would be appreciated. Thank you.

score 0 · Answer 1 · answered Dec 07 '16 at 15:47

Reading about puma and workers it seems unclear to me what magic is puma doing while forking workers, etc. So ended up deciding to create a rake task for it:

rails g task mynamespace process_queue

Then put into lib/tasks/mynamespace.rake the code needed to start processor:

namespace :mynamespace do
  desc "Process reuqest queue"
  task process_queue: :environment do
    MyConcern.process_op_queue
  end
end

Then start this as a separate process instead of a separate app thread. This makes it possible to run as many processors and in as many containers as I like. The good thing with the rake task is that it has all behavior as in the app itself. That means MyModel.something would automatically work as inside the app.

nohup bin/rails mynamespace:process_queue > my.log

Rails 5 starting thread from initializer gone crazy

1 Answers1