Premium Queue package for handling jobs and messages in NodeJS.
Questions tagged [bull.js]
31 questions
22
votes
5 answers
Bull Queue Concurrency Questions
I need help understanding how Bull Queue (bull.js) processes concurrent jobs.
Suppose I have 10 Node.js instances that each instantiate a Bull Queue connected to the same Redis instance:
const bullQueue = require('bull');
const queue = new…
Brian
- 563
- 1
- 5
- 15
13
votes
2 answers
How can I make sure that a job doesn't run twice in Bull?
I have two functions, scheduleScan() and scan().
scan() calls scheduleScan() when there's nothing else to do except scheduling a new scan, so scheduleScan() can schedule a scan(). But there's a problem, some jobs run twice.
I want to make sure that…
salep
- 1,332
- 9
- 44
- 93
13
votes
1 answer
nest bull separate process for queues and api
I have a nestjs application which is exposing a few REST APIs. One of the APIs triggers a job which processes some tasks. The problem is that when the job gets triggered the application stops serving REST requests which leads to health check…
Rahul Sharma
- 5,562
- 4
- 24
- 48
8
votes
1 answer
How to listen to the completed event in bull queue - just for the current job
When running the code below it prints too many results.
My suspicion is that the completed event listens to all the previous jobs made in the current queue instance.
How can I manage the completed event to listen just to the current job…
Da-Vi
- 141
- 1
- 1
- 8
6
votes
1 answer
How to merge namespace has no exported interface in TypeScript
I am using the queue lib Bull in TypeScript. Its definition is:
node_modules/@types/bull/index.d.ts
declare const Bull: {
(queueName: string, opts?: Bull.QueueOptions): Bull.Queue;
// something like above
};
declare namespace Bull: {
…
Brick Yang
- 5,388
- 8
- 34
- 45
5
votes
1 answer
Redis master/slave setup on Kubernetes throwing error: BRPOPLPUSH { ReplyError: MOVED 2651
I'm using the excellent Redis based Bull.js as a job queue on Kubernetes.
It's configured as a cluster:
When Kubernetes restarts upon deployments, I run into this following error:
BRPOPLPUSH { ReplyError: MOVED 2651 :6379
at parseError…
bob_cobb
- 2,229
- 11
- 49
- 109
5
votes
2 answers
Bull queue: Ensure unique job within time period by using partial timestamp in jobId
I need to ensure the same job added to queue isn't duplicated within a certain period of time.
Is it worth including partial timestamps (i.e. D/M/Y-HH:M) in my unique jobId strings, so it processes only if not in the same Minute?
It would still…
alyx
- 2,593
- 6
- 39
- 64
4
votes
2 answers
Node JS: Passing Response Object to Bull Queue for Server Side Events
I am stuck at an architectural decision. I have Node + Express app, It has an API to upload files. After the upload is done, the response is closed and uploaded file is processed by FFMPEG batch wise with the help of Bull Queue + Redis. This…
Abhinandan Chakraborty
- 861
- 2
- 15
- 23
3
votes
1 answer
Is it ok to ignore the promise returned by the process method of a bull queue?
I use the bull library to set up a processing queue, and when I use the process method for an existing queue I get a warning since this method returns a promise. I noticed bull's documentation ignores this return value and I want to know if it's…
Julia Suarez
- 65
- 1
- 7
3
votes
1 answer
Use NodeJS Bull Queue seperate processor
I'm trying to run the process in different processor file itsef as its in bull documentation, I've added the file as below.
// -------- Queue.js ----------
formatQueue.process(__dirname + "/processors/format-worker.js");
// On Cmplete…
rakcode
- 2,256
- 4
- 19
- 44
3
votes
2 answers
Job processing microservices using bull
I would like to process scheduled jobs using node.js bull. Basically I have two processors that handle 2 types of jobs. There is one configurator that configures the jobs which will be added to the bull queue using cron.
The scheduler will be in one…
Ganesh Kumar
- 3,220
- 1
- 19
- 27
3
votes
2 answers
How can server 1 know reliably when server 2 has completed a specific job? (bull.js)
Server 1 is calling this function:
let job = await transactions.add({ foo: 'bar' });
let status = await job.finished();
console.log(status);
Server 2 is consuming the job as follows:
transactions.process(QueueOptions.maxConcurrent, async (job,…
hitwill
- 575
- 1
- 9
- 25
2
votes
1 answer
How to gracefully shutdown bullmq when running inside an express server?
I have an express app which uses bullmq queues, schedulers and workers. Even after pressing Ctrl + C I can still see the node process running inside my Activity manager but my server on the terminal shuts down. I know this because the bullmq task…
PirateApp
- 5,433
- 4
- 57
- 90
2
votes
2 answers
Bull.js jobs stalling despite timeout being set
I have a Bull queue running lengthy video upload jobs which could take any amount of time from < 1 min up to many minutes.
The jobs stall after the default 30 seconds, so I increased the timeout to several minutes, but this is not respected. If I…
Simon
- 123
- 3
- 9
2
votes
0 answers
Whats the quickest way to iterate through failed Bull.js jobs and delete them properly?
I have a problem with my video upload pipeline getting stalled and users looking at a "waiting" notification all the time. It is a video upload pipeline that converts the videos with ffmpeg and uploads to AWS. It seems that there are failed/stuck…
lovgrandma
- 133
- 1
- 9