shoryuken error to `validate_queues': The specified queue(s)

Question

I am using rails "Shoryuken" gem but I am getting errors for validation on queues on my development environment when I started rails server below is the error:-

gems/shoryuken-2.0.11/lib/shoryuken/environment_loader.rb:172:in `validate_queues': The specified queue(s) ["development_worker"] do not exist (ArgumentError)

I have used below settings:-

config/shoryuken.yml

aws:
  access_key_id: <%= ENV["SQS_IAM"] %>
  secret_access_key: <%= ENV["SQS_IAM_SECRET"] %>
  region: <%= ENV["SQS_IAM_REGION"] %>
concurrency: 25  # The number of allocated threads to process messages. Default 25
delay: 0         # The delay in seconds to pause a queue when it's empty. Default 0
queues:
  - ["<%= Rails.env %>_worker", 1]

initializers/shoryuken.rb

def parse_config(config_file)
  if File.exist?(config_file)
    YAML.load(ERB.new(IO.read(config_file)).result)
  else
    raise ArgumentError, "Config file #{config_file} does not exist"
  end
end

config = parse_config([Dir.pwd, 'config/shoryuken.yml'].join('/')).deep_symbolize_keys
Shoryuken::EnvironmentLoader.load(config)

I want the queue should be environment specific.

MatayoshiMariano · Accepted Answer · 2017-08-09T13:13:31.787

1

Ravindra, looking at the code of shoryuken, you are getting the error because you do not have created an SQS queue named development_worker, is that rigth?

You will need to create one queue for each developer, because shoryuken will be running in the computers of each developer. If you don't do that, all shoryuken processes of each computer will be polling messages of the same queue. Imagine that there are two shoryuken processes, sh1 and sh2, that correspond to the dev1 and dev2 machine respectively. If the dev1 sends a SQS msg to the dev-queue, the sh2 proccess can be able to poll the message that the dev1 sent.

If you wish to avoid to create the queues in AWS, you can take a look at this.

edited Aug 09 '17 at 13:13

answered Aug 09 '17 at 12:42

MatayoshiMariano

2,026
19
23

yes, I have not created queue "development_worker" at AWS SQS. and I have tried to create a queue by using bundle exec Shoryuken sqs create development_worker and it runs fine but after running the rails server in the development environment it through same error. could you please help me I am not familiar with Shoryuken, i have used sidekiq. – Ravindra Yadav Aug 09 '17 at 12:48
Have you enter to [aws sqs](https://console.aws.amazon.com/sqs) and see the queue created? – MatayoshiMariano Aug 09 '17 at 12:50
after creating a queue at "aws sqs" it works fine but our team doesn't want to create the different-2 queue for each developer at "aws sqs". – Ravindra Yadav Aug 09 '17 at 12:53
@RavindraYadav I have updated the answer regarding to your question. And the error can still happen if you do not have the same keys of aws or the corresponding IAM users for the developers. – MatayoshiMariano Aug 09 '17 at 13:01
Thank you for your explanation, Could you please tell any way to setup Shoryuken without creating a queue at "aws sqs" in rails local environment. – Ravindra Yadav Aug 09 '17 at 13:09
Your welcome! I edited the answer to provide you a link on how to mock the queue in AWS. – MatayoshiMariano Aug 09 '17 at 13:14
1

If you are using Rails Active Job, in development you can use the inline adapter, which won't require Shoryuken nor SQS queues, then you can configure the Shoryuken adapter only for production. If you want to use Shoryuken in development, I recommend creating queues per developer `ENV['USER']` might be a good prefix. SQS does not charge per queue, it charges per requests. If you want to use Shoryuken in development, go with multiple queues. – Pablo Cantero Aug 26 '17 at 22:07

shoryuken error to `validate_queues': The specified queue(s)

1 Answers1