Node.js seems to not free memory after loop

Question

const axios = require('axios');
const JSDOM = require('jsdom').JSDOM;

axios.get('https://facebook.com')
    .then(response => {
        let i=1;
        while (true) {
            console.log(i++);
            const dom = new JSDOM(response.data);
            dom.window.close();
        }
    });

The above will run (on my machine) 440 times, then crash with the following:

<--- Last few GCs --->

[13411:0x5dc70d0]    63566 ms: Mark-sweep 2042.3 (2051.1) -> 2041.3 (2051.1) MB, 2407.4 / 0.0 ms  (average mu = 0.123, current mu = 0.027) allocation failure scavenge might not succeed
[13411:0x5dc70d0]    67327 ms: Mark-sweep 2042.0 (2051.1) -> 2041.1 (2050.9) MB, 3745.2 / 0.0 ms  (average mu = 0.055, current mu = 0.004) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x145cc79]
    1: StubFrame [pc: 0x145dab5]
Security context: 0x0d2e98240921 <JSObject>
    2: /* anonymous */ [0x374dcc798739] [/pathtomyproject/node_modules/parse5/lib/tokenizer/index.js:~644] [pc=0x323d7cd98d24](this=0x10b528b73279 <Tokenizer map = 0x242f38ec5c11>,95)
    3: getNextToken [0x374dcc799419] [/pathtomyproject/node_modules/parse5/lib/tokenizer/i...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Writing Node.js report to file: report.20200215.163602.13411.0.001.json
Node.js report completed
 1: 0xa9d570 node::Abort() [node]
 2: 0xa9f832 node::OnFatalError(char const*, char const*) [node]
 3: 0xc0758e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xc07909 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xdb5e15  [node]
 6: 0xdb64a6 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]
 7: 0xdc4d19 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
 8: 0xdc5b55 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xdc862c v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
10: 0xd8f204 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
11: 0x10dc52e v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
12: 0x145cc79  [node]
Aborted (core dumped)

So I guess even after reassigning dom on each iteration, the previous values are still being stored on the heap. Why is this and how do I prevent it?

`while(true)` that's not a good idea. :). Try using some sort of async loop, otherwise your not giving the GC any breathing space. — Keith, Feb 15 '20 at 18:38
I found [this explanation of GC](https://stackoverflow.com/a/52745907/1233305) helpful. Especially the part about "Garbage collection is never done immediately (at least not in V8)." Which reaffirms what @Keith says about breathing space. — David784, Feb 15 '20 at 18:43
@David784 if you read the entirety of that answer you linked, you'd realize that it _contradicts_ what Keith says about "breathing space". The GC needs no manually-provided "breathing space". In fact, you can see on the crashing stack trace that the GC did run, it just wasn't able to free anything. So this looks like a leak, and would need further investigation find out where the bug is. — jmrk, Jul 24 '22 at 12:32

jfriend00 · Answer 1 · 2020-02-15T21:30:49.067

Garbage collection in V8 is done when your code is not running and is in between doing things. Your while(true) loop runs forever and thus you never give V8 a change to do garbage collection.

For a lot of additional reasons, you never do a forever running loop in Javascript (unless you have an await inside the loop) because nodejs is a largely single-threaded event driven system and when you are busy cranking on a while(true) loop, NO events can ever get processed so lots of things can't get done in nodejs.

Why is this and how do I prevent it?

Garbage collectors are often designed to let your code run and not interfere with their execution and then, in a moment when your code isn't doing anything and has finished with its internal state for what it was doing, the garbage collector examines the state of objects in the heap to find objects that are no longer in use.

You prevent this by not going in a tight loop like this without ever providing any slack time that the GC can run. As explained above, a tight loop like this prevents any other events in nodejs from running anyway so it's generally a bad reason for other reasons too.

Note, that your code not only creates 444 dom instances and all the objects associated with those instances, but it doesn't provide any ability to clean up all the temporary variables/strings that were used in the creation of that dom instance which could be substantial when parsing HTML so the total memory usage is significantly higher than just the 444 finished dom instances. If, you insert any type of breathing room into the loop, then the GC will get a chance to clean things up that are no longer being used.

While I agree that long/endless loops are usually not a good idea, I disagree that this can explain the OOM crash here. There is no way to (accidentally or intentionally) prevent the GC from running. So this issue needs investigation to find the cause of the memory leak. A Heap Snapshot might provide crucial insight. — jmrk, Jul 24 '22 at 12:36

Node.js seems to not free memory after loop

1 Answers1