NodeJS Bull queue.add stringify data causing errors

Question

I am trying something like:

queue.add('queue', {
      someId: '123',
      service: this.service, // a service injected into the class where queue.add is present
    }, { // other opts });

I get the error:

(node:94191) UnhandledPromiseRejectionWarning: TypeError: Converting circular structure to JSON
    at JSON.stringify (<anonymous>)
    at Job.toData (<project>/node_modules/bull/lib/job.js:192:20)
    at addJob (<project>/node_modules/bull/lib/job.js:80:23)
    at queue.isReady.then (<project>/node_modules/bull/lib/job.js:94:14)
    at process._tickCallback (internal/process/next_tick.js:68:7)
(node:94191) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)

which I believe is coming from https://github.com/OptimalBits/bull/blob/develop/lib/job.js#L190-L200 .

Seems like the default is to try and stringify the entire job object, including the data field. So it complains on data fields provided that cannot be fully stringified. Is there no way around this? Is there no way to add a job to a queue with non-stringifiable custom data fields?

The reason I am injecting the service object instance into the job is because I am using a separate process in nestjs and there is no dependency injection available. I need access to that service in order to carry out certain DB operations from within the process. I don't want to create a separate DB connection every time a process runs.

Any help would be appreciated, thank you.

EDIT: I think my assumption here is incorrect. Redis itself cannot store non-stringified values.

score 0 · Answer 1 · answered Jun 10 '21 at 19:32

Seems like the default is to try and stringify the entire job object, including the data field. So it complains on data fields provided that cannot be fully stringified. Is there no way around this?

You can exclude properties, but that's probably not what you want (the service property would be ignored).

The reason I am injecting the service object instance into the job is because I am using a separate process in nestjs and there is no dependency injection available

What about sending an id that matches the service? Something like this.service.id

I need access to that service in order to carry out certain DB operations from within the process. I don't want to create a separate DB connection every time a process runs

Maybe your service could expose an API of some sort that your workers could consume, so that all the db communication still occurs at service level?

Hi @Anthony Garcia-Labiad thanks for your response. Can you clarify what you mean by > Maybe your service could expose an API of some sort that your workers could consume, so that all the db communication still occurs at service level? How does that help? The problem is that the workers would not even have access to the service. > sending an id that matches the service? Something like this.service.id I don't think there is an easy way to do this since hte services are all in memory — Arjun Mehta, Jun 10 '21 at 19:51

NodeJS Bull queue.add stringify data causing errors

1 Answers1