Upload file using NodeJS and BusBoy

Question

I am uploading a file using NodeJS. My requirement is to read the stream into a variable so that I can store that into AWS SQS. I do not want to store the file on disk. Is this possible? I only need the uploaded file into stream. The code I am using is(upload.js):

var http = require('http');
var Busboy = require('busboy');

module.exports.UploadImage = function (req, res, next) {

    var busboy = new Busboy({ headers: req.headers });

    // Listen for event when Busboy finds a file to stream.
    busboy.on('file', function (fieldname, file, filename, encoding, mimetype) {

        // We are streaming! Handle chunks
        file.on('data', function (data) {
                // Here we can act on the data chunks streamed.
        });

        // Completed streaming the file.
        file.on('end', function (stream) {
            //Here I need to get the stream to send to SQS
        });
    });

    // Listen for event when Busboy finds a non-file field.
    busboy.on('field', function (fieldname, val) {
            // Do something with non-file field.
    });

    // Listen for event when Busboy is finished parsing the form.
    busboy.on('finish', function () {
        res.statusCode = 200;
        res.end();
    });

    // Pipe the HTTP Request into Busboy.
    req.pipe(busboy);
};

How do I get the uploaded stream?

score 1 · Answer 1 · answered Jul 29 '17 at 22:50

1

On busboy 'file' event you get parameter named 'file' and this is a stream so you can pipe it.

For example

busboy.on('file', function (fieldname, file, filename, encoding, mimetype) { 
    file.pipe(streamToSQS)
}

answered Jul 29 '17 at 22:50

Whisnesky

26
2

score 1 · Answer 2 · answered Aug 11 '17 at 06:36

I hope that will help you.

busboy.on('file', function (fieldname, file, filename, encoding, mimetype) {
    var filename = "filename";
    s3Helper.pdfUploadToS3(file, filename);
  }
busboy.on('finish', function () {
        res.status(200).json({ 'message': "File uploaded successfully." });
    });
    req.pipe(busboy);

score 0 · Answer 3 · answered Oct 22 '18 at 13:48

While the current and existing arguments assume one could actually just send the stream (file) off to something that can receive the stream, the actual chunks are received in the file callback methods you implemented.

From the docs: (https://www.npmjs.com/package/busboy)

file.on('data', function(data) {
   // data.length bytes seems to indicate a chunk
   console.log('File [' + fieldname + '] got ' + data.length + ' bytes');
});
      
   
file.on('end', function() {
  console.log('File [' + fieldname + '] Finished');
});

Update:

Found the constructor docs, second argument is a readable stream.

file(< string >fieldname, < ReadableStream >stream, < string >filename, < string >transferEncoding, < string >mimeType) - Emitted for each new file form field found. transferEncoding contains the 'Content-Transfer-Encoding' value for the file stream. mimeType contains the 'Content-Type' value for the file stream.

Upload file using NodeJS and BusBoy

3 Answers3