Stack Overflow
Stack Overflow

Questions tagged [beanstalkd]

Beanstalkd is a simple, fast work queue.

Beanstalkd is a simple, fast work queue.

Its interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously.

401 questions
10
votes
4 answers

What do I use when a cron job isn't enough? (php)

I'm trying to figure out the most efficient way to running a pretty hefty PHP task thousands of times a day. It needs to make an IMAP connection to Gmail, loop over the emails, save this info to the database and save images locally. Running this…
mike
  • 2,722
  • 6
  • 31
  • 46
10
votes
0 answers

Supervisor on Heroku

I use a remote server to process beanstalkd queues that I would like to use on my application running on Heroku. Is there any way to run Supervisor to monitor that the queue command (Laravel: php artisan queue:listen) is running?
csm232s
  • 1,660
  • 4
  • 32
  • 60
9
votes
2 answers

Multiple queue listeners will run the same job over multiple processes

I have a simple web application that is written with the Laravel 4.2 framework. I have configured the Laravel queue component to add new queue items to a locally running beastalkd server. Essentially, there is a POST route that will add an item to…
Tim Groeneveld
  • 8,739
  • 3
  • 44
  • 60
8
votes
1 answer

Laravel queue with supervisord opening too many FIFO files

I'm running a Laravel queue job called "webhooks" using beanstalkd and supervisord on my Ubuntu server. I can see the job properly running with process ID 4403: webhooks RUNNING pid 4403, uptime 4 days, 19:47:01 As you…
flyingL123
  • 7,686
  • 11
  • 66
  • 135
8
votes
2 answers

Laravel Queue Failed Jobs

I was trying to record some data from other table when the jobs fails. It works great in failed jobs table but I cant get the Queue::failing(function($connection, $job, $data) to work every time the job failed. I did try to put it in global.php but…
Yuusha
  • 175
  • 1
  • 3
  • 10
7
votes
1 answer

Distributed video encoding - Gearman vs Beanstalkd

Im looking to build a distributed video encoding cluster of a few dozen machines. Ive never worked with a messaging queue before, but the 2 that I started playing around with were Gearman and Beanstalkd. Beanstalk seems to be a lot simpler and…
user15063
7
votes
3 answers

beanstalkd - what happens to reserved, but not completed jobs?

I've created a PHP script that reads from beanstalkd and processes the jobs. No problems there. The last thing I've got to do is just write an init script for it, so it can run as a service. However, this has now raised another question for me. When…
manavo
  • 1,839
  • 2
  • 14
  • 17
7
votes
1 answer

PHP/Beanstalkd: Spawning Multiple workers in parallel

I've got a script which checks which MX record belongs to an email address. I've got about 300k emails to check. So a single threaded process is going to take a long time. I've got beanstalkd with a queue and php is sending emails to it via a…
PaulM
  • 3,281
  • 3
  • 28
  • 38
6
votes
0 answers

Laravel: Synchronisch queue system

I am trying to set up an API which uses a queue system on another server to handle the requests. Let me start what I am trying to accomplish without the queue system (no authorization to keep it simple): Using Postman for example making a GET…
Ruben Colpaert
  • 91
  • 10
6
votes
2 answers

Laravel exception handler for queue worker

I am having a weird issue while throwing the exception inside my beanstalkd job class. When I throw a simple exception (throw new \Exception();) outside the job class, it calls the report() method in App\Exceptions\Handler and I can do my tweaks…
Tibin Paul
  • 806
  • 9
  • 23
6
votes
1 answer

golang - bufio read multiline until (CRLF) \r\n delimiter

I am trying to implement my own beanstalkd client as a way of learning go. https://github.com/kr/beanstalkd/blob/master/doc/protocol.txt At the moment, I am using bufio to read in a line of data delimited by \n. res, err :=…
Gravy
  • 12,264
  • 26
  • 124
  • 193
6
votes
1 answer

Preserve beanstalkd queue on restart or crash

I'm using beanstalkd to managed queues. I just realised that if there are jobs in a queue and the beanstalkd process is restarted or crashes then the job is lost forever (or so I think). Is there a way to preserve the jobs in the queue on beanstalkd…
Wasim
  • 4,953
  • 10
  • 52
  • 87
6
votes
3 answers

How to check if job in in Beanstalkd queue has been completed in Laravel

I need to check if job added to queue (Beanstalkd) has been completed in Laravel (Laravel 5) and in case it's completed I need to return updated record (task added to queue updates record in database). I've added to my…
Marcin Nabiałek
  • 109,655
  • 42
  • 258
  • 291
6
votes
2 answers

Simple scalable work/message queue with delay

I need to set up a job/message queue with the option to set a delay for the task so that it's not picked up immediately by a free worker, but after a certain time (can vary from task to task). I looked into a couple of linux queue solutions…
idevelop
  • 199
  • 2
  • 10
6
votes
1 answer

Celery or beanstalkd or both?

I know (but I do not understand) that Celery can use Beanstalk as delivery mechanism. From the Beanstalkd Homepage "Beanstalk is a simple, fast work queue." From the Celery Homepage "It’s a task queue with focus on real-time processing, while also…
blueFast
  • 41,341
  • 63
  • 198
  • 344
1
2
3
…
26 27