11

We run tests in separate processes and a few of our test suites contain more than 20 files. How can we limit (instead of increase) the amount of memory a Node.js process uses to less than say 300MB? If we don't limit memory, we could use up to 20x500MB, which is about 10GB, which is too much memory.

I can find a few articles online about increasing memory, but I can't find anything on limiting memory per process.

Specifically, we are using the core child_process module to create child processes and I guess we need some flag or arg to pass to the cp's to designate a memory cap of sorts.

Alexander Mills
  • 90,741
  • 139
  • 482
  • 817
  • 1
    https://groups.google.com/forum/#!topic/nodejs/IYsQ_bXlzcg contains such a phrase: "2) use --max-old-space-size flag to set the memory limit *lower* or higher". Did you try this parameter? – Dzenly Dec 04 '15 at 07:02
  • 1
    Actually I see few ways to decrease memory: 1) rewrite code to use less memory, to avoid memleaks and more effectively use GC, 2) more often GC using (--max-old-space-size), 3) set quotes from OS to kill processes which exceeded them. – Dzenly Dec 04 '15 at 07:10
  • I tried this node --max_old_space_size=100 and it worked, then I tried node --max_old_space_size=10 and it worked, and then I tried node --max_old_space_size=1 and it bombed, so it seems to work :) – Alexander Mills Dec 04 '15 at 07:18
  • ok, then I will form some answer from my thoughts, It is not so usefull, but let it be :) – Dzenly Dec 04 '15 at 07:55

1 Answers1

7

Useful links:

nodejs decrease v8 garbage collector memory usage

https://groups.google.com/forum/#!topic/nodejs/IYsQ_bXlzcg

https://www.quora.com/How-does-Node-js-do-memory-allocation

Options to be considered:

–nouse-idle-notification

–expose-gc + gc() function from your code.

–max-old-space-size

–max-new-space-size

--max-semi-space-size

Also to decrease memory, less memory consuming algorithms can be used, the code can be more GC friendly. Also native C++ modules can help, but check them thoroughly for memory leaks and errors with some valgrind.

Also quotes from OS can be applied to kill processes which exceed some memory limits.

I know, without some concious option descriptions this answer is not so usefull, but I not very good learned them for now, so this "answer" is just a direction.

Community
  • 1
  • 1
Dzenly
  • 1,611
  • 12
  • 18
  • thanks :) I am also looking for a way, given these node options, to pass these options when forking a child_process in Node – Alexander Mills Dec 04 '15 at 19:55
  • this https://nodejs.org/api/child_process.html does not make it clear how or if it is possible – Alexander Mills Dec 04 '15 at 19:55
  • 1
    It seems node.js has not fork like POSIX one, i.e. node.js always require executable and allows to specify executable arguments. So executable will be "node" and arguments will be: --max-bla-bla your_script.js. Also I investigated how to force node.js to use certain options at running certain scripts. Someone downvoted my topic, but it helped me. http://stackoverflow.com/questions/32798347/how-to-set-nodejs-interpreter-arguments-from-javascript-source/32804439#32804439 – Dzenly Dec 04 '15 at 20:18