Piping readable stream using superagent

Question

I'm trying to create a multer middleware to pipe a streamed file from the client, to a 3rd party via superagent.

const superagent = require('superagent');
const multer = require('multer');

// my middleware
function streamstorage(){
    function StreamStorage(){}

    StreamStorage.prototype._handleFile = function(req, file, cb){
        console.log(file.stream)  // <-- is readable stream
        const post = superagent.post('www.some-other-host.com');

        file.stream.pipe(file.stream);

        // need to call cb(null, {some: data}); but how
        // do i get/handle the response from this post request?
    }
    return new StreamStorage()
}

const streamMiddleware = {
    storage: streamstorage()
}

app.post('/someupload', streamMiddleware.single('rawimage'), function(req, res){
    res.send('some token based on the superagent response')
});

I think this seems to work, but I'm not sure how to handle the response from superagent POST request, since I need to return a token received from the superagent request.

I've tried post.end(fn...) but apparently end and pipe can't both be used together. I feel like I'm misunderstanding how piping works, or if what i'm trying to do is practical.

Answer 1

Superagent's .pipe() method is for downloading (piping data from a remote host to the local application).

It seems you need piping in the other direction: upload from your application to a remote server. In superagent (as of v2.1) there's no method for that, and it requires a different approach.

You have two options:

The easiest, less efficient one is:

Tell multer to buffer/save the file, and then upload the whole file using .attach().

The harder one is to "pipe" the file "manually":

1. Create a superagent instance with URL, method and HTTP headers you want for uploading,
2. Listen to data events on the incoming file stream, and call superagent's .write() method with each chunk of data.
3. Listen to the end event on the incoming file stream, and call superagent's .end() method to read server's response.