Sidekiq configuration for multiple environments

Question

I have looked at multiple sources and tried various scenarios but couldn't resolve this hence the issue. Please point me in the right direction.

Like everybody I have 3 env (development, staging and production).

I have the following in my sidekiq.yml file

# Options here can still be overridden by cmd line args.
#   sidekiq -C config.yml  
---
:verbose: false
:namespace: xyz
:logfile: log/sidekiq.log
:concurrency:  25
:strict: false
:pidfile: tmp/pids/sidekiq.pid
:queues:
  - [stg_xyz_tests_queue, 10]
  - [stg_default_xyz_queue, 2]
  - [stg_xyz_default_queue, 3]
development:
  :verbose: true
  :concurrency:  15
  :queues:
    - [dev_xyz_queue, 10]
    - [dev_default_xyz_queue, 2]
    - [dev_xyz_default_queue, 3]
staging:
  :queues:
    - [stg_xyz_queue, 10]
    - [stg_default_xyz_queue, 2]
    - [stg_xyz_default_queue, 3]
production:
  :queues:
    - [prod_xyz_queue, 10]
    - [prod_default_xyz_queue, 2]
    - [prod_xyz_default_queue, 3]

With this I was hoping that when I start sidekiq with the command

RAILS_ENV=#{rails_env} bundle exec sidekiq -C config/sidekiq.yml

that it would pickup all the values from the configuration file and start sidekiq with the appropriate queues and log file at log/sidekiq.log but that doesn't work. Sidekiq starts but it only creates the stg_xyz_tests_queue, stg_default_xyz_queue and stg_xyz_default_queue no matter what environment we use.

The other approach I tried was using the following code in the config/environments/development.rb

  #configure Sidekiq for dev environment
  Sidekiq.configure_server do |config|
    config.options[:namespace] = "xyz"
    config.options[:concurrency] = 25
    config.options[:verbose] = true
    config.options[:strict] = false
    config.options[:logfile] = "log/sidekiq.log"
    config.options[:pidfile] = "tmp/pids/sidekiq.pid"

    queues = Array.new
    10.times do
      queues.push "dev_xyz_queue"
    end

    2.times do
      queues.push "dev_default_xyz_queue"
    end

    3.times do
      queues.push "dev_xyz_default_queue"
    end

    config.options[:queues] = queues
    puts "Sidekiq server config options for development => #{config.options.to_yaml}"
  end

With this the queues are created ok but the logfile is not created or written and I need to duplicate this code for all the 3 environments.

What is the best way to get sidekiq working seamlessly for my setup Thanks for your help in advance !!!

[link](https://github.com/mperham/sidekiq/issues/758) Sidekiq site has the same question listed as an issue — user1687078, Mar 07 '13 at 00:22

score 20 · Answer 1 · answered Oct 23 '13 at 11:25

20

Use -e option

bundle exec sidekiq -e beta -C config/sidekiq.yml

If all environments(development, staging and production) are on same server then use namespace. In your initializers/sidekiq.rb file,

Sidekiq.configure_server do |config|
    config.redis = { url: 'redis://localhost:6379/0', namespace: "sidekiq_app_name_#{Rails.env}" }
end

Sidekiq.configure_client do |config|
    config.redis = { url: 'redis://localhost:6379/0', namespace: "sidekiq_app_name_#{Rails.env}" }
end

answered Oct 23 '13 at 11:25

Ranjithkumar Ravi

3,352
2
20
22

1

And what if they are not all on the same server? – Mark Murphy Feb 20 '14 at 15:37
Then its not necessary to set the namespace here. – Ranjithkumar Ravi Feb 27 '14 at 13:44
1

I meant how would you define the sidekiq config in initializers/sidekiq.rb if you were using different servers for different environments? – Mark Murphy Feb 28 '14 at 14:14
I assume, you are using different servers for different environments. On each server, you have installed Redis & Sidekiq. So the config would be config.redis = { url: 'redis://localhost:6379/0'} – Ranjithkumar Ravi Mar 04 '14 at 04:52
2

Just for curiosity, what is that '/0' in the end of the URL? – Ziyan Junaideen May 16 '16 at 07:42
1

@ZiyanJunaideen the `/0` is for the database. Some folks encourage to use separate db's on the same server or even separate Redis instances. Read this and avoid namespaces: https://www.mikeperham.com/2015/09/24/storing-data-with-redis/ – vysogot Feb 19 '21 at 14:23

score 9 · Answer 2 · answered Mar 07 '13 at 19:49

9

Use -e to pass the environment.

bundle exec sidekiq -e production -C config/sidekiq.yml

Thanks to mperham for the answer.

answered Mar 07 '13 at 19:49

user1687078

1,353
3
10
10

`RAILS_ENV=production bundle exec sidekiq` should also work I guess ? – Cyril Duchon-Doris Nov 29 '18 at 14:53

score 0 · Answer 3 · answered Aug 11 '16 at 13:17

0

Use to set log, and environment support:

bundle exec sidekiq -d -L log/sidekiq.log -e production -C config/sidekiq.yml

answered Aug 11 '16 at 13:17

Darpan Chhatravala

470
5
16

Sidekiq configuration for multiple environments

3 Answers3

Linked