Reading larger sftp files with ssh2 sftp Node

Question

I'm trying to use the ssh2-sftp library to read/write a file in Node. When I do an sftp.get on a larger CSV file (but not too large -- like 2 MB only) on an sftp site, and then read data on the returned stream, the call is hanging on me after the 14th stream.on("data") call. I've tested this with a few different sample files and the code works fine on smaller files. But if a CSV file is big enough to get past that 14th call, it just hangs and it's like it can't read anymore even though there's more there to read. And the stream.on("close") never gets called into either.

Obviously this is pretty weird behavior. Was hoping maybe somebody has run into something similar using this library and had some guidance.

If it helps at all, here is some code

   sftp.get(currentFileName).then((readStream) => {
    var counter = 0;

    readStream.on("data", function(d) {
      counter++;
      console.log("counter = " + counter);
    });

    readStream.on("close", function(err) {
      if (err) {
        console.error("Problem with read stream for file " + currentFileName + ", error = ", err);
      }
      //done reading the individual file, all went well
      else {
        console.log("done reading the stream");
      }
    }); 

    readStream.on('error', function(e){
      console.error("Error retrieving file: " + e);
    })

    readStream.resume();

And after the 14th call into readStream.on("data"), it just freezes up. With maybe half the file read.

score 1 · Accepted Answer · answered Oct 11 '16 at 13:41

The issue turned out to be that ssh2-sftp seems to be running an outdated version of the underlying ssh2 library. Switching from ssh2-sftp to the most recent (0.5.2) version of ssh2 and using that library directly fixed the issue (which might have been this one: https://github.com/mscdex/ssh2/issues/450)

Reading larger sftp files with ssh2 sftp Node

1 Answers1