4

I've set up Laravel (5.5) queuing with Beanstalkd, and I'm using Supervisor. Laravel docs mention the following:

By pushing jobs to different queues, you may "categorize" your queued jobs and even prioritize how many workers you assign to various queues.

but it's never mentioned how to do that. If I have 2 queues on my beanstalkd connection, and I do the following:

php artisan queue:work beanstalkd --queue=high,low

That will always process ALL of the jobs from high before it processes anything from low, if I'm understanding the docs correctly:

To start a worker that verifies that all of the high queue jobs are processed before continuing to any jobs on the low queue, pass a comma-delimited list of queue names to the work command

Let's say my queues are called apprequests and emails. How would I go about adding 1 dedicated worker for emails queue which won't care about the apprequests queue, and then 4 workers for the apprequests queue?

Here's my Supervisor laravel_worker.conf config:

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/my_app/artisan queue:work beanstalkd --queue=apprequests --timeout=330
autostart=true
autorestart=true
user=root
numprocs=4
redirect_stderr=true
stdout_logfile=/var/www/my_app/worker.log

Am I right to assume that I can just add another Supevisor config, for example laravel_email_worker.conf, with this command - command=php /var/www/my_app/artisan queue:work beanstalkd --queue=emails --timeout=30 and set numprocs to 1, and that should dedicate 1 worker to my emails queue which will process jobs from that queue regardless of the apprequests queue, or am I going all wrong about this?

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
GTCrais
  • 2,039
  • 2
  • 26
  • 32

1 Answers1

3

Yes, you can do as you wrote. You can set up one process that will look at email queue jobs and 4 processed that will look at apprequests queue.

But you might be also interested in Laravel Horizon that uses Redis as queue driver and can be set up to auto balance queue to increase number of workers for certain queues depending on current load.

Marcin Nabiałek
  • 109,655
  • 42
  • 258
  • 291
  • Thanks for the answer. I wanted to use Horizon, but I read up on Redis, and apparently it had some issues which I didn't want to deal with (https://github.com/laravel/framework/issues/8577 - note that last reply is September 23, 2017), so I went with Beanstalkd and it's working like a charm. – GTCrais Nov 16 '17 at 00:00
  • 1
    @Marcin Nabialek, if i want to use one different worker for each API endpoints i need to call ( as on each endpoints, different calls with different parameters are executed within around 10mi) and that i have more than 50 API endpoints, is it necessary to create 50 worker manually or it could be done programmatically? – LearningPath May 21 '18 at 11:35