0

I have a service, what receive a post request with a file and json data too. I use the body-parser package either in the app.js. I want to send the file to a "filer" service, and process the answer from that, but I don't want to pipe the request, because I need to process the json content too and make some actions after the filer answered.

const Busboy = require('busboy');
const request = require('request');

const sendFile = (req, file, callback) => {
  return request.post({
    uri: 'http://localhost:5000/stream',
    headers: req.headers,
    formData: { value: file }
  }, (err, resp, body) => {
    if (err) return callback(err);
    return callback();
  });
};

app.post('/route', (req, res, next) {
  const busboy = new Busboy({ headers: req.headers });

  busboy.on('file', (fieldName, file) => {
    file.on('error', err => reject(err));
    return sendFile(req, file, (err, link) => {
      file.resume();
      if (err) return reject(err);
    });
  });

  busboy.on('field', (fieldName, val) => {
    // process the json here...
  });

  busboy.on('finish', () => {
    console.log('busboy.on(finish)');
    return next();
  });

  req.pipe(busboy);
}

The filer service the following: 

app.post('/stream', (req, res, next) => {
  const busboy = new Busboy({ headers: req.headers });
  // here we are ok
  busboy.on('file', function (fieldName, file, name) {
    // but this part never run
    res.send(200, { fileId: fileDoc._id });
  });

  return req.pipe(busboy);
});

Unfortunatelly the filer service never answer, and I don't know, where is a missing part. Tried to put the file.resume() to some places inside the busboy.on('file'), but doesn't helped.

kree
  • 440
  • 1
  • 9
  • 26

1 Answers1

0

Its probably because the file stream from busboy is never processed properly by request formData.

Another way is:

  1. Temporarily write the stream to a local file in main service.(using fs.createWriteStream)
  2. Create a stream from that file and pipe it to filer service.(using fs.createReadStream)
  3. Do whatever processing you need to do in main service.
  4. Wait for response from filer service and call next()

This way you can even use the file in main service if you need to or send the file to another service.

Sundeep Narang
  • 59
  • 1
  • 4
  • I try to avoid to store the files on local, this load is unneccesary. Mostly because I don't have anything to modify in the file. – kree Sep 28 '17 at 07:24
  • I believe the issue with stream from busboy being used in formdata. formData for requestjs, probably isn't recognizing it as a valid file stream. I have had this issue. The only other thing you could do is pipe this stream to request directly and then on close event callback. – Sundeep Narang Sep 28 '17 at 19:52