2

I'm using the promise-ftp Node.js library to download a file from an FTP server. Function ftpClient.get(...) returns a Promise of ReadableStream, data of which should be accessible using data and end events, however, I encounter a weird behaviour that is not described in the documentation.

//Some code that returns a to-be-downloaded file's name
.then((fileName) => {
    return ftpClient.get(fileName)
})
.then((fileStream) => {            
    let chunks = [];

    return new Promise((resolve, reject) => {
        fileStream.on('data', (chunk) => {
            chunks.push(chunk);
        });
        fileStream.on('end', () => {
            resolve(Buffer.concat(chunks).toString('utf8'));
        });
    });
});

This is a solution on how to read a stream into a variable that I derived from answers to this question. However, it doesn't work: the data and end events are just never emitted.

Comparing my code to this example in library documentation, the only difference is that the ReadableStream is piped somewhere beforehand.

Now, if I add fileStream.pipe(...),

//...
.then((fileStream) => {

    fileStream.pipe(fs.createWriteStream('foo.txt'));
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    let chunks = [];

    return new Promise((resolve, reject) => {
//...

everything works perfectly.

So my question is: what exactly is the purpose of adding a .pipe() and how is it connected to data and end events?

Rafael Sofi-zada
  • 511
  • 3
  • 14

1 Answers1

0

Pipe is the correct way to read data from a stream. The idea comes from Linux Pipes. Something like:

fileStream | stdout

Here fileStream is being pipe'd to stdout

on is to register an event listener i.e. if some part of the program has the pipe set up, you can listen in to the same.

Not streaming any data when no one is reading it (i.e. its not being piped to anything) is an obvious optimisation (it could have other constraints too).

Yes I understand the mentioned answer can be misleading. Office node docs on streams is better. And there is a new upward trending better answer.

Vedant Agarwala
  • 18,146
  • 4
  • 66
  • 89