1

I have some files stored on a CDN server which is not to be directly accessed from client. So I proxy the requests via the public accessible server running ExpressJS and use request module to fetch the data server-side and return it in response.

It is working and in code looks something like this:

var request = require('request');
var app = express();
var internalUrl = 'https://my.storage-cdn.com/private/info/file.xml';

app.get('/somefile.xml', function (req, res) {
    request(internalUrl).pipe(res);
});

The issues I faced with above method are:

  • the storage/cdn server appends some response headers of its own which include some private information and as such can be a security issue when exposed in response. And above method of piping the res object to request doesn't remove those headers. It passes those headers as is to response. I want to remove those headers.
  • I want to add some eTag and cache-control headers so the file could get cached properly.

I have tried changing it to something like this:

app.get('/somefile.xml', function (req, res) {
    request(internalUrl, function (err, response, body) {
        if (!err && response.statusCode == 200) {
            res.writeHead(200, {...});   // write custom headers I need
            res.end(body);
        }
    });
});

This allows me to overwrite the headers to my liking, but in this method I have to wait for whole file to get downloaded on the server side first before I start sending the bytes in my response and with some files being as large as 1MB, it really affects the response time adversely.

So my question is - is there a way to not have to wait for whole file to download on server side before start sending response but still be able to manipulate response headers?

codneto
  • 2,319
  • 3
  • 24
  • 36

1 Answers1

4

You can hook onto the 'response' event:

const SECRET_HEADERS = ['Set-Cookie', 'X-Special-Token']

app.get('/somefile.xml', function (req, res) {
  request(internalUrl).on('response', function (response) {
    SECRET_HEADERS.forEach(function (header) {
      response.removeHeader(header)
    })
  }).pipe(res)
})
idbehold
  • 16,833
  • 5
  • 47
  • 74
  • Thank you. That did the trick for me. Would you know if 'response' event is fired when the data is the stream starts becoming available, or when all http data is loaded from the remove server. I am trying to understand how streams work and how this is different from a 'data' event. – codneto Mar 03 '17 at 20:07
  • @codneto the 'response' event fires after the headers and HTTP status have been received and before any 'data' events fire. – idbehold Mar 03 '17 at 21:56