Stack Overflow
Stack Overflow

Questions tagged [bullmq]

BullMQ - Premium Message Queue for NodeJS based on Redis

BullMQ is a Node.js library that implements a fast and robust queue system based on Redis. The library is designed so that it will fulfill the following goals:

  1. Exactly once queue semantics, i.e., attempts to deliver every message exactly one time, but it will deliver at least once in the worst-case scenario*.
  2. Easy to scale horizontally. Add more workers for processing jobs in parallel.
  3. Consistent.
  4. High performant. Try to get the highest possible throughput from Redis by combining efficient .lua scripts and pipelining.
77 questions
0
votes
0 answers

Bull Arena for bullmq - TypeError: as of 3.0.0, bull-arena requires that the queue constructors be provided to Arena

I'm using bullmq not bull library import BullMQ from 'bullmq'; import Arena from 'bull-arena'; const arenaConfig = Arena({ BullMQ, queues: bullQueues, }); got an error TypeError: as of 3.0.0, bull-arena requires that the queue constructors be…
Vzans
  • 91
  • 1
  • 6
0
votes
0 answers

My Bullmq working not running tasks for all jobs in the queue

I am running a background job to generate pdfs using puppeteer and i am emailing out these pdfs using recipients name and email. below is my worker logic const sendCertificatesWorker = new Worker( queueName, async job => { const { data: { id, name,…
Olanitori Tobi
  • 1
0
votes
0 answers

how to get jobId for failed jobs that are moved from stalled jobs on bullMQ?

I was trying to run jobs which were CPU intensive and I was getting a "Error: job stalled more than allowable limit" error. I tried increasing the lock time but with increased load, the error persisted. I then used worker threads and wrapped the…
Ashfaq Ur Rahman N
  • 302
  • 2
  • 8
0
votes
0 answers

tsc-watch kills redis connection between app and redis server

I am trying to get ts-watch to work on a nodejs/typescript project which is using bullmq. The application is a simple API which does some long running calculations. These calculations are by the cueue manager bullmq. Right now I am having the…
Hannes F
  • 339
  • 2
  • 11
0
votes
0 answers

How to remove failed retrial jobs using BullMQ?

I use BullMQ to run jobs on the background. I have defined main job using retry strategy. I want to make sure that when main job fails and when retrying this job fails it removes retried failed jobs automatically. As I calculate error threshold and…
Yalchin Mammadli
  • 444
  • 4
  • 11
0
votes
0 answers

How can I efficiently scale CPU-intensive tasks using BullMQ Queue and child processes while overcoming limitations with object passing?

I am utilizing a machine learning (ML) object for object detection in images and have incorporated the use of the bullMQ queue. During testing on a few hundred images, I encountered slow performance and therefore attempted to enhance its…
Amir Alam
  • 155
  • 1
  • 12
0
votes
0 answers

```Bull queue error: ReplyError: MOVED 4913 dev-.use2.cache.amazonaws.com:6379``` while connecting with adding and processing BULL queue```

I tried to run BULL queue with aws redis cluster. But i am facing this error Bull queue error: ReplyError: MOVED 4913 dev-.use2.cache.amazonaws.com:6379 while connecting with adding and processing BULL queue``` I have tried adding hash before…
Divyanshu Sen
  • 71
  • 9
0
votes
0 answers

How to handle repeatable jobs in Bull.js when the queue is paused

I am using Bull.js with repeatable jobs. I want to be able to pause the queue while I'm updating the server. I was planning to use pause method of Bull.js but what I realized is that once the queue is unpaused Bull.js only runs the repeatable jobs…
Ismail Gül
  • 119
  • 12
0
votes
0 answers

how can I configure redis in cpanel for nodejs app

Actually I'm try to build app that can support queue for email sending and I'm using bull queue package. We all know it require redis connection in background to run bull queue. But here didn't find any resource to connect redis in cPanel for nodejs…
Nawaraj Jaishi
  • 459
  • 3
  • 12
0
votes
1 answer

bullMQ waitUntilFinished function not working

I'm creating a job using BullMQ and using waitUntilFinished function - it's working perfect on my local machine, but it's not working on my dev server and throw an error job wait credits_transaction_queue timed out before finishing, no finish…
Олег Шубин
  • 25
  • 6
0
votes
0 answers

Redis ETIMEDOUT gitlab ci

After added bullmq to the project stack, I started to face with problem in gitlab CI pipelines, to be more concrete in the integration tests, looks like something wrong with redis because application cannot connect to the redis - ioredis library…
mxxnseat
  • 314
  • 1
  • 2
  • 8
0
votes
1 answer

Graceful shutdown of BullMQ workers prevents Node JS from exiting

I follow the recommendation here and also this thread and I use the following code in order to gracefully shutdown BullMQ workers in my development environment on Ubuntu: const handleTermination = async () => { log.info(`Gracefully shutting…
user9839468
  • 31
  • 1
  • 5
0
votes
0 answers

Agenda job processing time slower than expected when running multiple jobs concurrently, causing backlog - any solutions?

I'm using the Agenda library for job scheduling in my Node.js application. However, I'm experiencing some unexpected behavior where jobs are scheduled for the wrong time, or not at all when I run a lot of jobs. For example, I use job.now() to…
Stefdelec
  • 2,711
  • 3
  • 33
  • 40
0
votes
0 answers

Redis Database Connection Error in BullMQ

I'm trying to create a queue object and connect it to my Redis enterprise database. Below is the code: import { Queue, FlowProducer } from "bullmq"; import env from "dotenv"; env.config() const redisOptions = { username: 'Admin_user', …
Jriffs
  • 125
  • 1
  • 8
0
votes
0 answers

Bull job data strips functions from variables

My intention is to pass the avatar a user uploads to a queue that can be processed when my app is idle. The avatar (file) has properties like mimetype, filename, etc. as well as a createReadStream function, but when I pass the avatar object to the…
maxspiri
  • 348
  • 2
  • 9
1 2 3
4
5 6