Why while loop is needed for reading a non-flowing mode stream in Node.js?

Question

In the node.js documentation, I came across the following code

const readable = getReadableStreamSomehow();

// 'readable' may be triggered multiple times as data is buffered in
readable.on('readable', () => {
  let chunk;
  console.log('Stream is readable (new data received in buffer)');
  // Use a loop to make sure we read all currently available data
  while (null !== (chunk = readable.read())) {
    console.log(`Read ${chunk.length} bytes of data...`);
  }
});

// 'end' will be triggered once when there is no more data available
readable.on('end', () => {
  console.log('Reached end of stream.');
});

Here is the comment from the node.js documentation concerning the usage of the while loop, saying it's needed to make sure all data is read

// Use a loop to make sure we read all currently available data
  while (null !== (chunk = readable.read())) {

I couldn't understand why it is needed and tried to replace while with just if statement, and the process terminated after the very first read. Why?

score 1 · Answer 1 · answered May 08 '21 at 11:48

From the node.js documentation

The readable.read() method should only be called on Readable streams operating in paused mode. In flowing mode, readable.read() is called automatically until the internal buffer is fully drained.

Be careful that this method is only meant for stream that has been paused.

And even further, if you understand what a stream is, you'll understand that you need to process chunks of data.

Each call to readable.read() returns a chunk of data, or null. The chunks are not concatenated. A while loop is necessary to consume all data currently in the buffer.

So i hope you understand that if you are not looping through your readable stream and only executing 1 read, you won't get your full data.

Ref: https://nodejs.org/api/stream.html

score 0 · Answer 2 · answered Apr 30 '23 at 03:32

Use a loop to make sure we read all currently available data

I couldn't understand why it is needed

const SIZE_IN_BYTES = 1;

process.stdin.on('readable', function () {
  console.log('Data available to read');
  let chunk;
  while ((chunk = process.stdin.read(SIZE_IN_BYTES)) !== null) {
    console.log(`Reading 1 byte ${chunk.toString()}`);
  }
});

In above example note that you can pass size(optional) argument to read method to specify how much data to read. If we have more data(>1 byte as per example) available to read then while loop will help to read more than 1 byte of data.

It reads the stream data repeatedly until there is no more data to read. It pauses the loop execution until new data becomes available in the stream buffer. read() will pause the loop execution(when return is null or when buffer is in empty state) until more data becomes available.

If the size argument is not specified, all of the data contained in the internal buffer will be returned.

When execution is in paused state and new data is available in buffer, readable event will be emitted.

and tried to replace while with just if statement, and the process terminated after the very first read. Why?

if statement executes the read() only once(buffer state is not emptied), and it doesn't pause the execution for more data to become available. So after the first chunk(1 byte in our example) is read, it stops executing.

Why while loop is needed for reading a non-flowing mode stream in Node.js?

2 Answers2