16

I am using node and am considering manually running garbage collection in node. Is there any drawbacks on this? The reason I am doing this is that it looks like node is not running garbage collection frequently enough. Does anyone know how often V8 does its garbage collection routine in node?

Thanks!

Bowen Su
  • 1,219
  • 3
  • 13
  • 15
  • 2
    What problem are you really trying to solve (show us relevant code) and how are you measuring the memory usage. I suspect you have a measurement problem rather than a garbage collection problem as garbage collection is run in node.js idle time and is usually only an issue if you have a really, really long thread of execution that creates lots of temporary objects. In a normal node.js server environment, there should be plenty of idle time (when using proper async programming). – jfriend00 Jun 05 '15 at 20:01
  • It may be worth reading this article: https://strongloop.com/strongblog/node-js-performance-garbage-collection/ – jfriend00 Jun 05 '15 at 20:03
  • I am using ps --sort -rss -eo rss,pid,command | head for checking memory. I am using an amazon server (free tier) so there is only 1 gig of memory. If I start to run multiple instances of my app on this and do not garbage collect manually, the memory usage skyrockets until the server crashes – Bowen Su Jun 05 '15 at 20:04
  • 1
    Just looking at total process memory does not tell you much about internal garbage collection. Memory freed by the garbage collector may or may not be immediately returned to the OS. Plus, a well written app will have caches and other dynamically allocated resources. You will need to look at the contents of the node.js heap to understand where memory is actually being used. – jfriend00 Jun 05 '15 at 20:06

3 Answers3

24

I actually had the same problem running node on heroku with 1GB instances.

When running the node server on production traffic, the memory would grow constantly until it exceeded the memory limit, which caused it to run slowly.

This is probably caused by the app generating a lot of garbage, it mostly serves JSON API responses. But it wasn't a memory leak, just uncollected garbage.

It seems that node doesn't prioritize doing enough garbage collections on old object space for my app, so memory would constantly grow.

Running global.gc() manually (enabled with node --expose_gc) would reduce memory usage by 50MB every time and would pause the app for about 400ms.

What I ended up doing is running gc manually on a randomized schedule (so that heroku instances wouldn't do GC all at once). This decreased the memory usage and stopped the memory quota exceeded errors.

A simplified version would be something like this:

function scheduleGc() {
  if (!global.gc) {
    console.log('Garbage collection is not exposed');
    return;
  }

  // schedule next gc within a random interval (e.g. 15-45 minutes)
  // tweak this based on your app's memory usage
  var nextMinutes = Math.random() * 30 + 15;

  setTimeout(function(){
    global.gc();
    console.log('Manual gc', process.memoryUsage());
    scheduleGc();
  }, nextMinutes * 60 * 1000);
}

// call this in the startup script of your app (once per process)
scheduleGc();

You need to run your app with garbage collection exposed:

node --expose_gc app.js
d4n3
  • 1,453
  • 14
  • 13
  • I'm really hoping this works because this has been bothering me for a week. I even upgraded to standard 2x only to find out the app would eventually eat up whatever amount of memory you give it – lwdthe1 Sep 14 '17 at 08:01
  • No luck, I think it actually worsened the problem because my app still ate memory and the manual garbage collection only stopped my app's responsiveness. – lwdthe1 Sep 14 '17 at 15:02
  • if someone tries to copy paste the command: it is --expose-gc rather than --expose_gc – younes zeboudj Mar 14 '22 at 12:22
6

I know this may be a bit of a tardy reply to help to OP, but i thought I would collaborate my recent experiences with Node JS memory allocation and garbage collection.

We are currently working on a node JS server running on a raspberry pi 3. Every so often it would crash due to running out of memory. I initially thought this was a memory leak, and after a week and a half of searching through my code and coming up with nothing, I thought the problem could have been exacerbated by the fact that Node JS allocates more memory than available on the Rpi3 for its processes before it does the GC.

I have been running new instances of my server with the following commands:

'node server.js --max-executable-size=96 --max-old-space-size=128 --max-semi-space-size=2'

This effectively limits the total amount of space that node is allowed to take up on the local machine and forces garbage collections to be done more frequently. Thus far, we are seeing a constant usage of memory and it confirms to me that my code was not leaking initially, but rather node was allocating more memory than possible.

EDIT: This link here outlines in more specific terms the issue I was dealing with.

-nodejs decrease v8 garbage collector memory usage -https://github.com/nodejs/node/issues/2738

Community
  • 1
  • 1
Henri De Boever
  • 89
  • 1
  • 5
4

V8 Run garbage collection when he thinks it's useful. There is no fixed delay for that. You can read this article to learn about garbage collection V8: https://strongloop.com/strongblog/node-js-performance-garbage-collection/

Anyway, it's a bad idea to run manually the garbage collector in your project because it blocks completely the node process. So during the garbage collection, your program won't handle any requests.

Sachacr
  • 704
  • 1
  • 9
  • 17