i am trying to find a way to wait for a buch of bullmq jobs and then when finished remove them. currently i do something like this:
const queues : Job[] = []
while (distance <= highZ + 1) {
queues.push( await extruderJobque.add(
`${distance}`,
{
folder, polygonFile, distance, skipMesh, weightsFile
},
{
removeOnComplete: true,
removeOnFail: true,
}
))
distance++
}
await new Promise((resolve, reject) => {
extrudeWorker.on('completed', async job => {
console.log('EXTRUSION JOB COMPLETED')
if (queues.map((id ) => id.id).includes(job.id)) {
const index = queues.map((id ) => id.id).indexOf(job.id);
await queues[index].remove()
queues.splice(index, 1);
console.log(`Removed job ${job.id} from queue`);
console.log("queue length: ", queues.length)
}
if (queues.length === 0){
resolve("")
}
// console.log(JSON.stringify(job.returnvalue))
})
extrudeWorker.on('failed', async job => {
console.log('JOB FAILED')
reject("")
})
})
the problem is, that
- does work if i have a concurrency of 1. when i have more then one process then it does not finish. and 2. when i run the algorithm 2 times, on "completed" returns also the old jobs. so first run it runs into on "completed" x times, when i run it again, it goes here x + 1 times and so on.
waht am i doing wrong here and how can i wait for the jobs?
thanks a lot!!
Cheers