How can I read and download a file from a ReadableStream on an API call (Node.js)?

Question

I am using ssh2-sftp-client to get files from a remote server, I'm running into an issue reading and downloading these files once I get() them.

At first, I was able to use the get() method to download the file when the API was hit - I could also return the whole file contents in a console.log statement then it started returning Buffer content. I updated with this:

npm install ssh2-sftp-client@3.1.0

And now I get a ReadbleStream.

function getFile(req,res) {
    sftp.connect(config).then(() => {
        return sftp.get(process.env.SFTP_PATH  + '/../...xml',true);
    }).then((stream)=>{
        const outFile = fs.createWriteStream('...xml')
        stream.on('data', (c) => {
            console.log(`Received ${c.length} bytes of data.`);
           outFile.write(c);  
           res.send('ok')         
          });
         stream.on('close', function() {    
        });
    }).catch((err) => {
        console.log(err, 'catch error');
    });

};

I have the above code that returns a stream but I'm not sure how to get the file - the write() method doesn't seem to work here.

Any advice or suggestions on how I can use this library to read and download files would be greatly appreciated

You can `pipe` the stream to the outFile stream – Stephen S Aug 21 '19 at 14:22 — Stephen S, Aug 21 '19 at 14:22

score 0 · Accepted Answer · answered Sep 17 '19 at 09:30

First, don't use version 3.x. That version has been deprecated. The most recent version is v4.1.0 and has had significant cleanup work to fix a number of small bugs.

If all you want to do is download the files, then use the fastGet() method. It takes 2 args, source path and destination path. It is a lot faster than plain get as it does the download in parallel.

If you don't want to do that, then the get() method has a number of options. If you only pass in one arg (source) it will return a buffer. If you pass in 2 args, the second arg must be either a string (path to local file) or a writeable stream. If a writeable stream, the data will be piped into that stream.

How can I read and download a file from a ReadableStream on an API call (Node.js)?

1 Answers1