SFTP in Node.js uploading empty files

Question

I'm having an issue with a project I'm working on whereby I'm downloading files from my FTP server using a node module called sftp-ssh2-client and then trying to pass the stream back into my FTP server but into a different directory.

The code looks like this:

const stream = sftp.get(`path/to/file.csv.gz`).then(() => {

}).catch((err) => {

});

sftp.put(stream, 'path/to/new/file.csv.gz').then(() => {

}).catch((err) => {

})

However, when I dial into my FTP server to the location where the file has been PUT to, the file size is 0 and when downloaded is corrupted. Can anyone tell me what I'm doing wrong here?

Many thanks,

G

robertklep · Accepted Answer · 2018-12-16T09:22:12.340

3

sftp.get() doesn't return a stream, it returns a promise that resolves to a stream, so your code should look something like this:

sftp.get('path/to/file.csv.gz').then(stream => {
  return sftp.put(stream, 'path/to/new/file.csv.gz');
}).catch(err => {
  ...
});

However, it seems to me that you could just use sftp.rename(), which wouldn't require downloading and uploading the entire file:

sftp.rename('path/to/file.csv.gz', 'path/to/new/file.csv.gz').then(...);

Also, if you do want to take the download-then-upload route, make sure you read the documentation regarding encoding.

edited Dec 16 '18 at 09:22

answered Dec 16 '18 at 09:07

robertklep

198,204
35
394
381

Hi Robert - I've implemented your solution but the files are still 0 bytes when I download them? – GeorgeBT1902 Dec 16 '18 at 10:16
Which encoding would I use for a .csv.gz file? I don't know where to look in the docs – GeorgeBT1902 Dec 16 '18 at 10:26
_"when downloading binary data, 'null' should be passed"_. Since GZ files are binary, encoding should be `null` – robertklep Dec 16 '18 at 10:27
I've now done this: sftp.get('path/to/file.csv.gz', false, null).then(stream => { but it still is an empty file – GeorgeBT1902 Dec 16 '18 at 10:47
1

Try writing the stream to a local file first: `sftp.get(...).then(stream => stream.pipe(fs.createWriteStream('localfile')))` – robertklep Dec 16 '18 at 10:58
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/185313/discussion-between-user9914176-and-robertklep). – GeorgeBT1902 Dec 16 '18 at 11:05

vasudev.p · Answer 2 · 2020-02-18T14:33:05.450

 function uploadFile(req, res) {
    req.pipe(req.busboy);
    req.busboy.on('file', function(fieldname, file, filename) {
        // debug("Uploading: " + filename);
        filename = (new Date()).getTime() + '-' + filename;

        //Path where file will be uploaded
        var fstream = fs.createWriteStream(FILE_STORE + filename);

        file.pipe(fstream);

        fstream.on('close', function() {
            //   debug("Upload Finished of " + filename);
            var newFileName = FILE_STORE + filename;
            if (!fs.existsSync(newFileName)) {

                res.status(404).json({ 'message': 'file not found' })
                return;
            }
            res.status(200).json({ file: filename });
        })
    })

}

SFTP in Node.js uploading empty files

2 Answers2