Stack Overflow
Stack Overflow

Questions tagged [bullmq]

BullMQ - Premium Message Queue for NodeJS based on Redis

BullMQ is a Node.js library that implements a fast and robust queue system based on Redis. The library is designed so that it will fulfill the following goals:

  1. Exactly once queue semantics, i.e., attempts to deliver every message exactly one time, but it will deliver at least once in the worst-case scenario*.
  2. Easy to scale horizontally. Add more workers for processing jobs in parallel.
  3. Consistent.
  4. High performant. Try to get the highest possible throughput from Redis by combining efficient .lua scripts and pipelining.
77 questions
2
votes
1 answer

BullMQ "error" event seems never triggered

There is code using BullMQ and an 'error' event. It seems never triggered. const queue = new BullMQ.Queue(this.queueName, { connection }); queue.on('error', handleError); queue.on('failed', handleJobError); Is this event type exists in BullMQ? It's…
Slim
  • 1,256
  • 1
  • 13
  • 25
2
votes
1 answer

Can we read queue data created by bullmq in nodejs with python

I am currently running application in node.js and need to write python code for OCR services but for communication i am using Bullmq right now. I need to Publish data from Bullmq into queue and consume that data using Python Is this possible ?. Or…
Muhammad Omer
  • 21
  • 1
2
votes
0 answers

BullMQ throwing "Missing lock for job failed" after using moveToDelayed

I'm following this pattern in the docs https://docs.bullmq.io/patterns/process-step-jobs But it seems like the example code throws a Missing lock for job failed error. Following is a minimal reproducible version import { Worker, Queue } from…
Sumit Ghosh
  • 1,033
  • 10
  • 29
2
votes
0 answers

How to emit events from BullMQ queue to only the user that is the owner of the job

My usecase of socket is not for chats, it's specifically to tell the front-end what BullMQ queue events happened, with a progress bar and telling when a job is done or failed. Currently when I'm emitting events, it's going for all users, I tried to…
Lucas Fernandes
  • 31
  • 2
2
votes
2 answers

Solution for user-specific background job queues

I have been researching how to efficiently solve the following use case and I am struggling to find the best solution. Basically I have a Node.js REST API which handles requests for users from a mobile application. We want some requests to launch…
Bernat Felip
  • 323
  • 5
  • 18
2
votes
0 answers

BullMQ and postgres connection (knex)

I have a worker running that make call to a Postgres database. Even though I'm initializing the connection to the database outside of the worker (with knex in a singleton), the number of connection are going to the roof when there is a lot of…
ldemesla
  • 81
  • 1
  • 1
2
votes
1 answer

It is possible to query a queue or add a propertie to check if a job with certain property already exists on queue in BULLMQ library

My question is simple. I just wanna to know if is possible to know is a job with certain propertie in jobData is already on BULLMQ queue. In docs there's nothing about this an didn't found anything about this situation on internet :/ Any info is…
NerckJF
  • 59
  • 5
2
votes
1 answer

What is the difference between BullMQ and message queue?

I have a user creation use-case: When user created, a welcome mail should be sent to him. In my current implementation, I have 2 services: user service (API) Notification service I created a "notification queue" using BullMQ. When new user is…
Tal
  • 23
  • 4
2
votes
1 answer

How to gracefully shutdown bullmq when running inside an express server?

I have an express app which uses bullmq queues, schedulers and workers. Even after pressing Ctrl + C I can still see the node process running inside my Activity manager but my server on the terminal shuts down. I know this because the bullmq task…
PirateApp
  • 5,433
  • 4
  • 57
  • 90
2
votes
1 answer

How do I measure progress while streaming a postgres database query via pg-query-stream?

I have an ETL query which requires me to read a large number of rows and then apply some transformation on them and save them back to a separate table in Postgres. I am using pg-query-stream and I plan to run the test function below inside a bullmq…
PirateApp
  • 5,433
  • 4
  • 57
  • 90
1
vote
0 answers

Why on server is not running when i use bullmq?

When i try to use this code it's not logging Hello, World also i noticed that my server is not runninng on localhost 3000 what i'm doing wrong? using bullmq trying to run a function after 10 seconds how to do that? using bullmq? import { Queue }…
cwecae
  • 43
  • 1
  • 6
1
vote
1 answer

Bull (redis) process stops consuming new messages in NodeJS

I'm initialising bull's process at the start of my worker (running via docker and k8s), and it continues to listen to the message against the configured queue (redis broker). Problem is, I have noticed that after some time (few hours, it's not some…
ShahtajK
  • 117
  • 10
1
vote
1 answer

BullMQ: How to attach one worker with multiple queues in nodejs using bullmq?

I have to utilize one worker for more than one Queue. Is this possible in bullmq? As I went through the documentation I found option to provide only one queue name for worker. Is there any way using that I can allocate single worker to process…
Lalit Kolate
  • 31
  • 1
  • 4
1
vote
0 answers

Using Bullmq sandboxed job with NX by building files separately

I am working on an nx monorepo and it has multiple projects within it. I have been working on server code to complete heavy tasks that may arise separately from the rest of the server process. To achieve this I am using bullmq to queue tasks and…
Brook MG
  • 601
  • 10
  • 20
1
vote
0 answers

Job does not get picked up by worker in bullmq

I a using bullmq with ioredis in an express server. I have about 5 queues for different tasks. But none of the jobs seem to be getting picked up by workers. I have checked the redis connections, and possible typoes in the queue names. What am I…
The_C1oud
  • 11
  • 1
  • 1
1
2
3 4 5 6