Remove headers from a piped HTTP stream

Question

To give a short example of what I want to achieve, imagine we have an HTTP server already serving a given request:

require('http').createServer(function(req, res) {
    var payload = new Buffer('Hello World\n', 'utf8');
    res.writeHead(200, {
        'Content-Type': 'text/plain',
        'Content-Length': payload.length,
        'Connection': 'Keep-Alive'
    });
    res.end(payload);
}).listen(8888);

Now, consider the existence of a second HTTP server that receives a request and in order to serve it it needs to invoke the first server. (A classical scenario when we have a web app that needs to invoke a given RESTful endpoint, for instance).

var http = require('http');
http.createServer(function(req, res) {

    var fwdrq = http.request({
        hostname: 'localhost',
        port: 8888,
        path: '/',
        method: 'GET'
    });

    fwdrq.on('response',function(response){
        response.pipe(res);
    });

    fwdrq.on('error', function(error){
        console.log(error);
    });

    fwdrq.end();

}).listen(9999);

Now, I like the idea of piping the original request with the response of the internal request done by the second server, that's pretty convenient and that's exactly what I need. But before I send the response back to client, I would like to get an opportunity to remove any hop-by-hop headers from the response sent from the first server. I want the entire payload for sure, and I want some of the headers in its response, but certainly not all.

For instance, I would like to avoid sending headers like Proxy-Authenticate or Connection, or any of those considered hop-by-hop headers. Also, I'd like to consider the possibility of not sending back keep alives if that's how I want my second server to operate, etc.

I know how to add headers before the response is delivered, but once piped, I have no clue how I could remove headers from the response being piped.

Don't get me wrong, I know I could do this by subscribing to the events and then building the response on my own, but I want to know if this is possible still if I am piping the responses.

Does anyone out there know how to pull off this trick?

score 3 · Accepted Answer · answered Mar 22 '15 at 20:43

From my testing with pipe, only the payload is getting piped and not the headers.

Open up Chrome developer console (Firebug on Firefox, Fiddler on desktop) and look at the HTTP response coming back from the server. The headers from the upstream server are not carried back to the end user. If you manually add headers at the proxy server, you'll see they do show up to the end user.

var http = require('http');

http.createServer(function(req, res) {
    var payload = new Buffer('Hello World\n', 'utf8');
    res.writeHead(200, {
        'Content-Type': 'text/plain',
        'Content-Length': payload.length,
        'Connection': 'Keep-Alive',
        'UpstreamHeader': 'Test'
    });
    res.end(payload);
}).listen(8888);

http.createServer(function(req, res) {

    var fwdrq = http.request({
        hostname: 'localhost',
        port: 8888,
        path: '/',
        method: 'GET'
    });

    //Uncomment the lines below to add headers from the proxy server
    /*res.writeHead(200, {
        'ProxyHeader': 'Test'
    });*/

    fwdrq.on('response', function(response) {
        response.pipe(res);
    });

    fwdrq.on('error', function(error) {
        console.log(error);
    });

    fwdrq.end();

}).listen(80);

Interesting, I had assume that the entire payload would be piped, but you're right, just body is being piped, the headers are ignored. I wonder when the Connection: keep-alive header is added to the response then. It is not at the beginning of the requests, but it is definitely reaching the browser. — Edwin Dalorzo, Mar 22 '15 at 20:49

Remove headers from a piped HTTP stream

1 Answers1

Linked