Stack Overflow
Stack Overflow

Questions tagged [beanstalkd]

Beanstalkd is a simple, fast work queue.

Beanstalkd is a simple, fast work queue.

Its interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously.

401 questions
2
votes
0 answers

Load Balancing job queue among disproportionate workers

I'm working on a tool to automatically manage a job queue (in this case, Beanstalkd). Currently, you must manually set the number of available workers to pull jobs off the queue, but this does not allow for spikes in jobs, or it wastes resources…
Jeremy Wilson
  • 509
  • 1
  • 4
  • 16
2
votes
2 answers

How to move a beanstalkd job in the ready queue to the front if the queue is very big and is taking a while to process?

I had a quick question in regards to a beanstalkd queue. So say I have 500,000 ready jobs in the beanstalk queue which are just waiting be processed and at the same time more jobs are being added to this queue. All of these jobs have the same…
user1828300
  • 81
  • 1
  • 7
2
votes
1 answer

NodeJS setInterval bad for performance?

I have a beanstalkapp worker that is made with nodejs what it does is there is a PHP application which does all the site stuff and if there are errors or issues or notifications or what ever it adds it to the beanstalkapp. The nodejs then needs to…
Steven
  • 13,250
  • 33
  • 95
  • 147
2
votes
3 answers

BeanStalkd on Solaris doesnt return anything when called from the python library

i am using Solaris 10 OS(x86). i installed beanstalkd and it starts fine by using command "beanstalkd -d -l hostip -p 11300". i have Python 2.4.4 on my system i installed YAML and beanstalkc python libraries to connect beanstalkd with python my…
Numenor
  • 1,686
  • 12
  • 20
1
vote
1 answer

Queueing: N producers to N consumers

The requirement is as follows: There are N producers, that generate messages or jobs or whatever you want to call it. Messages from each procuder must be processed in order and each message must be processed exactly once. There's one more…
shylent
  • 10,076
  • 6
  • 38
  • 55
1
vote
1 answer

zookeeper queue delay?

What would you guys suggest to be a good way to implement a queue in zookeeper that has the ability to delay a job without blocking a worker? Reference beanstalkd delayed job option.
schone
  • 562
  • 4
  • 13
1
vote
0 answers

Is there a Zend_Queue_Adapter_Beanstalkd?

Before going to write my own Adapter I am looking for an existing Zend_Queue Adapter that supports beanstalkd. Is there a Open Source Adapter/Implementation somewhere for Beanstalkd in the Zend Framework? So far I could not find something on the…
favo
  • 5,426
  • 9
  • 42
  • 61
1
vote
1 answer

Best solution for running multiple intensive jobs at specific times

We have a web app that uses IMAP to conditionally insert messages into users' mailboxes at user-defined times. Each of these 'jobs' are stored in a MySQL DB with a timestamp for when the job should be run (may be months into the future). Jobs can be…
Mark Perkins
  • 260
  • 2
  • 10
1
vote
1 answer

Java based task processing server

Looking for a base framework for a small, lightweight server that can efficiently pull jobs/messages off of something like beanstalk or SQS and process them. Octobot (https://github.com/cscotta/octobot) looks good, but no one seems to be…
Josh Nankin
  • 2,518
  • 4
  • 28
  • 45
1
vote
0 answers

Clear beanstalk queue (laravel)

Project on larvel in Kubernetes I need clear queue My process /var/www $ ps PID USER TIME COMMAND 1 www-data 0:41 {supervisord} /usr/bin/python3 /usr/bin/supervisord -c /etc/supervisor/supervisord.conf -n 346 www-data 0:00 /bin/sh …
Dmitry
  • 11
  • 1
1
vote
1 answer

How to consume from beanstalk tube

I am using beanstalk to put messages in the tube and i want to consume it from the new EC2 instance that i will launch at runtime. I am able to put message in the tube but i am not able to consume it by any means. I performed the below steps to…
Rohan0980
  • 47
  • 5
1
vote
1 answer

Beanstalkc Timeout Question

I am using beanstalkc in Python for a queuing process for a program which has to parse a list of URLs. Hence I am using timeout in beanstalk for avoiding huge time consumption by any URL. But even after using it my process doesn't time out in the…
Srikanth
  • 21
  • 2
1
vote
1 answer

How to use beanstalkc in Python to queue URLs and perform jobs

I have a function named spider which takes seed as an argument. seed is the name of the URL I send to the spider function. Now my question is how do I use beanstalkc in Python to queue the URLs and perform the jobs.
Srikanth
  • 21
  • 2
1
vote
1 answer

Best way to deploy ruby beaneater background processing in production

I have been using beaneater for background processing in my RoR application. I am using upstart to run beaneater in the background by running a rake task from my upstart srcipt. exec bundle exec rake RAILS_ENV=production bs:beaneater And my rake…
chetang
  • 198
  • 3
  • 19
1
vote
1 answer

Beanstalkd tube missing

Integrated Pheanstalk library in my web app. Created three tubes for three type of jobs. Created a beanstalkd web console also to see the job status. All are working fine. The issue is, today when I checked the console, one of the tubes is missing.…
suneeth shihab
  • 43
  • 6
1 2 3
…
26 27