Connect.js middleware to rewrite request body

Question

I have an Express-based Node.js application. I would like to convert line ending characters in my incoming request before the request passes to the rest of my Connect middleware stack (specifically, convert all LF characters to CRLF characters). Is there an existing Connect.js middleware for rewriting the request body? Ideally, I'd like to be able to do something like the following:

var rewriter = bodyRewriter(function(bodyString){
   return bodyString.replace(/\n/g, "\r\n");
});
app.use(rewriter);

I've looked through this list of middleware but couldn't find anything that works. There's connect-modrewrite but it seems to work only for URLs, and not for request bodies; There's connect-injector but it only works for responses, not requests. I am very new to Node.js and would really like to avoid reinventing the wheel for this seemingly straightforward task... is there anything out there that I can use?

robertklep · Accepted Answer · 2013-06-07T07:55:05.013

This might do the trick:

var rewriter = function(req, res, next) {
  req.bodyString = '';
  req.on('data', function(chunk) {
    req.bodyString += chunk.replace(/\n/g, '\r\n');
  });
  next();
});
app.use(rewriter);
app.use(express.bodyParser());

In your routes, you'll have req.bodyString available.

However, this isn't going to work for any items that you access through req.body, because bodyParser is unaware of req.bodyString's existence.

And since bodyParser wants to read the request's body data directly, you'll have to take a different route and implement a middleware that processes req.body items to make the replacement:

app.use(express.bodyParser());
app.use(function(req, res, next) {
  for (var key in req.body) {
    req.body[key] = req.body[key].replace(/\n/g, '\r\n');
  }
  next();
});

EDIT: okay, after reading your comment and some thinking, this seems to work for clients that post their MIME messages with LF only. It's very much a hack though, so there are bound to be big issues with it.

The following code should be used instead of express.bodyParser():

app.use(express.json());
app.use(express.urlencoded());
app.use(express.multipart({ defer : true }));

app.use(function(req, res, next) {
  req.form.on('error', function(err) {
    console.error('form error', err);
    next();
  });
  req.form.on('end', function() {
    next();
  });

  var oldwrite    = req.form.write.bind(req.form);
  req.form.write  = function(buffer) {
    var string = buffer.toString();
    // shortcut: if a message contains `\r\n`, assume it's okay.
    if (string.indexOf('\r\n') === -1) {
      string = string.replace(/\n/g, '\r\n');
    }
    return oldwrite(new Buffer(string));
  };
});

Unfortunately neither of your solutions worked. Perhaps I should explain my scenario more clearly... I have a buggy client (which I have no control over) that sends LF instead of CRLF in MIME messages. This causes express.bodyParser to fail as soon as it gets to the MIME boundary (which is terminated by LF instead of CRLF as per the MIME standard). So I need to perform the substitution *before* the body gets to bodyParser. I did end up finding a solution, I'll post it as an answer, would really appreciate your comments on it. — Eugene Osovetsky, Jun 07 '13 at 01:22
Actually, turns out the "solution" I found doesn't work at all, so I still would very much appreciate some help with this... — Eugene Osovetsky, Jun 07 '13 at 02:26
Okay, that's going to be tricky, since I don't think you can process the body data in such a way that `bodyParser` can pick up on it. Let me think about it some more... — robertklep, Jun 07 '13 at 05:31
Thank you so much - it works now! (I had to use conncet.multipart instead of express.multipart) — Eugene Osovetsky, Jun 07 '13 at 09:08

Connect.js middleware to rewrite request body

1 Answers1