Node.js event loop ignores setTimeout inside read stream data events?

Question

I am clarifying a doubt that I have regarding how Node.js executes timers. Normally when having a setTimeout block in your node script, the event loop, even if does not have anything else will wait for specified milliseconds in setTimeout and exits once it has run the callback.

But consider below code

const fs = require('fs');

let readable = fs.createReadStream('large.csv');
readable.on('data', async chunk => {
    let rows = parseCsvRow(chunk);  
    try {
        for (let i = 0; i < rows.length; i++) {
            console.log(i);
            await someAsyncOperation(rows[i]);
        }
    } catch (error) {
        //something went wrong
    }
});

readable.on('end', chunk => {
    console.log('csv file processed successfully');
});

function someAsyncOperation(){
    return new Promise((resolve, reject) => {
        setTimeout(resolve, 2000);
    });
}

The csv file I am reading has about 300 rows and I am firing an ajax request for each(code hidden for simplicity) which takes about 100ms to run and i have simulated it here using a setTimeout.

When I run above program it exits almost instantly without waiting for all pending setTimeout timers to run successfully. Just because the timers where scheduled from inside a data event on read stream does event loop ignore it ? Should not it wait for all timers to execute ?

Note - I understand that above code is not the right way to do what I am doing but this is just a sample to demonstrate the concept that I am having a hard time understanding.

T.J. Crowder · Answer 1 · 2020-03-15T17:33:26.157

0

This has nothing to do with Node.js, or the event loop.

The problem is that you're passing an async function into something that won't do anything with the promise it returns. That's one of the promise antipatterns. Only use async functions as callbacks when one of these is true:

The thing executing the callback will use or return the promise (for instance, Promise.all because it uses them, or map on arrays because it returns them)
You don't care if the thing calling your async function continues working while your async function may be paused on an await, and you wrap everything in the async function in a try/catch that handles rejections.

The Node.js specific part is: If you want to prevent further calls to your data handler until you've finished doing something, you need to pause the stream and then resume it later.

edited Mar 15 '20 at 17:33

answered Mar 15 '20 at 17:04

T.J. Crowder

1,031,962
187
1,923
1,875

1

I understand point no 1 but not point no 2. I am returning promise from `someAsyncOperation` (it's there in the sample code) and resolving the promise in setTimeout. Why will it not wait for that ? – Vinay Mar 15 '20 at 17:17
@Vinay - Wow, I **completely** read past it. Yes, absolutely, there's a promise right there in the code. Wow, talk about not reading properly, sorry about that! I've edited the answer to remove the incorrect part. :-) – T.J. Crowder Mar 15 '20 at 17:33
1

@t-j-crowder that is fine. No problem. – Vinay Mar 15 '20 at 17:37

score 0 · Answer 2 · answered Mar 15 '20 at 17:43

As it is in most cases, my confusion was because of a typo in my sample code.

It was a typo in my code due to which there was an uncaught error in someAsyncFunction. This uncaught error made node to exit the program. I have fixed it and now it works as expected. The current code as in shown in the question description correctly waits for timers to be executed before node exits the process.

TL;DR - Node will wait for all timers to be executed before exiting the process. It does not matter if the timers are set from events callbacks or any other place for that matter(also you should not be having any uncaught errors)

Node.js event loop ignores setTimeout inside read stream data events?

2 Answers2