Sidekiq async tasks are done synchronously

Question

I have two Sidekiq workers:

# app/workers/client_worker.rb
class ClientWorker
  include Sidekiq::Worker

  def perform(*args)
    puts "client started\n"
  end
end

and

# app/workers/server_worker.rb
class ServerWorker
  include Sidekiq::Worker

  def perform(*args)
    i = 0
    loop do
      begin
        i = i + 1
        puts "server waiting"
        sleep(2)
        if i > 5
          break
        end
      rescue => e
      end
    end
  end
end

I have two Rails apps, same rails version, same ruby version, same gems versions.

rvm use ruby-2.4.2@rails5.2

On a fresh Rails App When I start server worker, then immediatly after, client worker, I get client message as expected amongst server messages:

2018-06-21T22:44:19.855Z 32364 TID-gn863tph0 ServerWorker JID-7cdd1fd3d6fe63c91d99f3b0 INFO: start
server waiting
server waiting
server waiting
2018-06-21T22:44:24.356Z 32364 TID-gn865q1zk ClientWorker JID-7dc887d6824d0b789afec69d INFO: start
client started
2018-06-21T22:44:24.357Z 32364 TID-gn865q1zk ClientWorker JID-7dc887d6824d0b789afec69d INFO: done: 0.001 sec
server waiting
server waiting
server waiting
2018-06-21T22:44:31.858Z 32364 TID-gn863tph0 ServerWorker JID-7cdd1fd3d6fe63c91d99f3b0 INFO: done: 12.003 sec

On an already developed Rails app

When I repeat this process, tasks are running one after the other:

2018-06-21T22:41:34.250Z 32247 TID-grdaosqf3 ServerWorker JID-5a2d9917e6575d6858644474 INFO: start
server waiting
server waiting
server waiting
2018-06-21T22:41:38.402Z 32247 TID-grdamch6b ClientWorker JID-5da83582991bf284f6357046 INFO: start
server waiting
server waiting
server waiting
client started
2018-06-21T22:41:46.259Z 32247 TID-grdaosqf3 ServerWorker JID-5a2d9917e6575d6858644474 INFO: done: 12.007 sec
2018-06-21T22:41:46.259Z 32247 TID-grdamch6b ClientWorker JID-5da83582991bf284f6357046 INFO: done: 7.858 sec

What I've tried

I have removed all installed gems in the old app, so that the old Rails app Gemfile ressembles the new one. I have removed all custom libraries, all jobs, all custom configurations
I have compared Sidekiq.options on both apps, they are the same:

{:queues=>[], :labels=>[], :concurrency=>25, :require=>".", :environment=>nil, :timeout=>8, :poll_interval_average=>nil, :average_scheduled_poll_interval=>5, :error_handlers=>[], :death_handlers=>[], :lifecycle_events=>{:startup=>[], :quiet=>[], :shutdown=>[], :heartbeat=>[]}, :dead_max_jobs=>10000, :dead_timeout_in_seconds=>15552000, :reloader=>#}

Now I'm running out of ideas! Can anyone suggest me a method to find out what's wrong?

EDIT Okey, after a long hesitation I decided to do the same change I made in new app in old app, namely: in new app I remarked that workers are not included automatically, so I added the following initializer:

# config/initializers/01_extensions.rb
Dir["#{Rails.root}/app/workers/*.rb"].each { |file| require file }

In old app, workers were included automatically (I really don't know which is the default behaviour!), so this initializer was not necessary.

BUT NOW when I decided to add the same initializer to old app, tasks are running async as expected.

Now I have two questions:

do we need to include sidekiq workers explicitly? (not mentionned in doc)
what doest the explicit including of workers change in the sidekiq behaviour?

score 2 · Answer 1 · answered Jun 21 '18 at 23:27

2

In development, Rails 5 effectively limits you to one job at a time, even with multiple threads. This global "executor" lock allows for the development reloading that Rails is famous for.

In production, jobs will be executed concurrently as you'd expect.

answered Jun 21 '18 at 23:27

Mike Perham

21,300
6
59
61

but I was able to run multiple jobs at time in dev mode, in a fresh rails app (provided that I add the initializer that I just mentioned in edit) – ubugnu Jun 21 '18 at 23:31

Sidekiq async tasks are done synchronously

1 Answers1