Environment specific shoryuken configuration

Question

I'm using the shoryuken gem by running

bundle exec shoryuken -R -C config/shoryuken.yml

and shoryuken.yml is

concurrency: 25
delay: 0
queues:
- [development_high,  30]
- [development_mid,   6]
- [development_low,   2]

This works fine. However, I'd like to use the environment specific queues, e.g. use production_high in production. I make sure these queues exist in

config/initializers/aws.rb using

%w(low mid high).each do |q|
  sqs.create_queue(queue_name: Rails.env + '_' + q)
end

Answer 1

One possible workaround is using

concurrency: 25
delay: 0
queues:
- [<%=ENV['RAILS_ENV']%>_high,  30]
- [<%=ENV['RAILS_ENV']%>_mid,     6]
- [<%=ENV['RAILS_ENV']%>_low,     2]

and start shoryuken with

RAILS_ENV=development bundle exec shoryuken -R -C config/shoryuken.yml

Answer 2

I usually use a specific config shoryuken.dev.yml in development.

For the queues in development, I strongly recommend to create a user (IAM) per developer, with the needed permissions for the resource "arn:aws:sqs:::${aws:username}-*", then in the config - [<%= aws_user_name %>_queue_name, 1].

If you are using ActiveJob, have a look at the inline option. Might be an alternative.

Last but not least, you can also add queues at runtime, check this.