0

Looking at Kue (https://github.com/LearnBoost/kue) I am somewhat confused whether I can just produce and consume jobs in the same node process or do I need to launch the consumer in a separate node process?

I need to offload some CPU-heavy tasks (generating PDFs) to a job queue without any HTTP request blocking.

ragulka
  • 4,312
  • 7
  • 48
  • 73

1 Answers1

0

I think you answered your own question. ;-) Yes -- your worker process(es) should be separate from the process serving HTTP requests, at least in production. In development, maybe you want to simplify your life and have everything run in one process, but even then I'd recommend separate processes.

danmactough
  • 5,444
  • 2
  • 21
  • 22
  • Then how would I exactly do it if I need to boot my whole app for processing the jobs? All the examples that I've seen produce and consume in the very same file/process. – ragulka Jan 21 '13 at 14:10
  • If your workers need the whole app there's nothing to do but have them boot the whole app. If that's an issue, then you have to design the workers to work without the app -- maybe do all the compute-intensive stuff then send results to main process via IPC. – danmactough Jan 21 '13 at 17:48
  • What would you say about spinning up the worker as a child process of the main app? Would it still be a bad idea? – ragulka Mar 09 '14 at 10:52
  • Child process is a fine idea! – danmactough Mar 09 '14 at 14:19