Stack Overflow
Stack Overflow

Questions tagged [bullmq]

BullMQ - Premium Message Queue for NodeJS based on Redis

BullMQ is a Node.js library that implements a fast and robust queue system based on Redis. The library is designed so that it will fulfill the following goals:

  1. Exactly once queue semantics, i.e., attempts to deliver every message exactly one time, but it will deliver at least once in the worst-case scenario*.
  2. Easy to scale horizontally. Add more workers for processing jobs in parallel.
  3. Consistent.
  4. High performant. Try to get the highest possible throughput from Redis by combining efficient .lua scripts and pipelining.
77 questions
1
vote
1 answer

BullMQ - add jobs to queues from different service to worker

I'm trying to figure out the best architecture for a scalable BullMQ implementation. We have a number of different services that are going to be feeding jobs into queues. In some situations we may have multiple different services feeding jobs into…
oldo.nicho
  • 2,149
  • 2
  • 25
  • 39
1
vote
1 answer

How to run cron job every minute in Queue using Workers with bullmq package in nodejs?

import { Queue, Worker, Job } from 'bullmq'; const myQueue = new Queue('firstjob', { connection: redis }); import { autorunJobs } from './processes/job.process'; async function addQueue() { await myQueue.add( …
Biku7
  • 450
  • 6
  • 16
1
vote
0 answers

key distribution is not correct in bull redis cluster

I have around 16 queues, which are being used for delayed jobs. Some queues are used multiple times, In a one-time slot, there are some queues which are being used around 50 times and the rest are 3 to 4 times. Implementation examples:- import Bull…
Shivam Raj
  • 23
  • 5
1
vote
0 answers

BullMQ - Group/Job based completion

I am using BullMQ, Redis and MySQL in a producer-consumer model to process jobs. I have a producer that looks like this: const jobOptions = { removeOnComplete: true, // remove job if complete delay: 60000, attempts: 3 }; const…
Sam
  • 602
  • 9
  • 21
1
vote
1 answer

How to check BullMQ queue size?

I have a BullMQ dynamic queue. Is there a way to check how many items there in queue? I already checked in google & stackoverflow and cannot find any solution.
Bohdan Srdi
  • 144
  • 1
  • 6
1
vote
1 answer

Bull job not getting triggered in cron[Node.js]

I have multiple background jobs processors set up using the bull package like below import { CronJob } from 'cron'; import Queue from 'bull'; let queue = new Queue('workers', { // settings: { lockDuration: 60 * 20000 }, defaultJobOptions:…
nipek
  • 810
  • 1
  • 9
  • 22
1
vote
2 answers

Bullmq doesn't pull job from a Redis over TLS

I'm using the very basic example (const queue = new Queue('Paint')) from the https://www.npmjs.com/package/bullmq page - everything works fine (it defaults to localhost:6379). However when I add a connection (new Queue('Paint', { connection }))…
deebugger
  • 145
  • 2
  • 8
1
vote
3 answers

remove specific repeatable jobs in bullmq (with jobid)

I need to remove specific individual job from queue. actual case is I've added a job into queue and then in some API call I need to remove that individual job from the queue. so, It won't be repeat in future. add: const job = await…
Harsh Makwana
  • 237
  • 3
  • 13
1
vote
2 answers

Repeatable jobs not getting triggered at given cron timing in Bull

I wanted to perform some data processing in parallel using Bull NPM and start processing each job at the given cron Time const Queue = require("bull"), /** * initialize the queue for executing cron jobs **/ …
Sagar Pednekar
  • 334
  • 4
  • 14
0
votes
0 answers

NestJs: How can I connect to a RedisCluster using Bull?

I have a globally available Redis Module which creates a RedisCluster connection which I am importing in AppModule. import { Module, Global } from '@nestjs/common'; import { Redis } from 'ioredis'; @Global() @Module({ providers: [{ …
code_optimizer
  • 55
  • 6
0
votes
0 answers

How to properly log errors from DocMQ ack.fail("error") call into MongoDB document error field?

Problem Even after calling await api.fail("Error: Some Error which needs to be stored in the error field of the doc"), it's failing the job but not logging the error in the DB. However, when the max retries count reached the limit, it call the…
Abhishek Jadhav
  • 1
  • 1
  • 2
0
votes
1 answer

Nestjs bullmq worker processor don't works with jest

Step to reproduce Create a new nest project : nest new nest-app Install dependencies : npm i @nestjs/bullmq ioredis Start a redis instance on default port 6379 Create this 2 files app.service.ts import { Injectable } from '@nestjs/common'; import…
TheSmartMonkey
  • 857
  • 1
  • 7
  • 23
0
votes
0 answers

How to retry succeeded job with bullmq

I am trying to restart a job in bull mq with using only redis-cli. The reason of not using JS api or just recreating a job is because redis is inside the k9s cluster on production and i basically unable to connect to it from outside. But i have…
Daniel Varentsoff
  • 1
  • 1
0
votes
0 answers

wait for bullmq jobs and remove them on finish / fail

i am trying to find a way to wait for a buch of bullmq jobs and then when finished remove them. currently i do something like this: const queues : Job[] = [] while (distance <= highZ + 1) { queues.push( await extruderJobque.add( …
Hannes F
  • 339
  • 2
  • 11
0
votes
0 answers

BullMQ: How to schedule and execute a job once at a specific timestamp in the future without using delay?

is there a built-in way in bullMQ to schedule a job to execute at a specific timestamp in the future? I know it can somehow be done with: const targetTime = new Date('03-07-2035 10:30'); const delay = Number(targetTime) - Number(new Date()); await…
yeln
  • 462
  • 2
  • 10
  • 23
1 2
3
4 5 6