Parse file content during file upload

Question

I was wondering if someone knows how to extract the file content from an uploaded file when using express/nodejs

I have the following code and it's clear how to pipe the input to a filestream, but how do I deserialize the upload to a plain javascript object? The purpose is to extract information from the uploaded file and use it somewhere else. Ideally I don't want to go via a temporary file on disk. The format of the file is json.

app.post("/upload", function(req, res){

    req.pipe(req.busboy);

    req.busboy.on('file', function (fieldname, file, filename) {

        console.log("Uploading: " + filename);
        console.log(file);
        console.log(fieldname);

        // I don't want to go via a file, but straight to a JS object


        //fstream = fs.createWriteStream('/files/' + filename);
        //
        //file.pipe(fstream);
        //
        //fstream.on('close', function () {
        //
        //    res.redirect('back');
        //
        //});
    });
});

The file upload is triggered like this:

var formData = new FormData($('form')[0]);

        e.preventDefault();

        $.ajax({
            url: 'upload',
            type: 'POST',
            data: formData,
            cache: false,
            dataType:'json',
            contentType: false,
            enctype: 'multipart/form-data',
            processData: false,
            success: function (response) {
            }
        });

Is there a reason why you can't either use `JSON.parse` as soon as you have all the data in memory using `file.on('data', ...) `, or using a module like [JSONStream](https://www.npmjs.com/package/JSONStream) — t.niese, Feb 08 '15 at 22:58
I have tried json.parse, but the file seems to be submitted as a string to the server and line break characters seem to cause a problem for the parser. — TGH, Feb 09 '15 at 03:54

score 1 · Accepted Answer · answered Feb 08 '15 at 23:10

1

If you don't mind buffering the file content first, you can do something like this:

app.post("/upload", function(req, res){
  req.pipe(req.busboy);
  req.busboy.on('file', function (fieldname, file, filename) {
    var buf = '';
    file.on('data', function(d) {
      buf += d;
    }).on('end', function() {
      var val = JSON.parse(buf);
      // use `val` here ...
    }).setEncoding('utf8');
  });
});

answered Feb 08 '15 at 23:10

mscdex

104,356
15
192
153

Then valid JSON is *not* being sent. Can you post example data of what is contained in `buf`? – mscdex Feb 09 '15 at 03:53
interesting... It looks like the whole issue was caused by inconsistent use of single vs double quotes in the json file – TGH Feb 09 '15 at 03:59
what is the use setEncoding here? Are we applying the setEncoding to the overall file object and then emitters are called? Or setEncoding will be called on the emitted data strings? – AnandShiva Oct 21 '20 at 10:34

Parse file content during file upload

1 Answers1