Why does https.Agent throw a parse error when proxying via node-http-proxy?

Question

I would like to enable the use of connection pooling via http.Agent and https.Agent in my node-http-proxy application. To do this I've established a secure agent, like so:

const secureAgent = new https.secureAgent({ keepAlive: true });

(I've elided the complementary http agent for brevity.)

I create a proxy server, as such:

const proxy = httpProxy.createProxyServer({});

Finally I proxy a request inside a Connect middleware like so:

proxy.web(req, res {
  agent: isSecure ? secureAgent : agent,
  target: ...,
  secure: false,
});

This seems to work for most requests, but once every few minutes I see an error that looks like this:

{
    "message": "Parse Error",
    "stack": "Error: Parse Error\n    at TLSSocket.socketOnData (_http_client.js:411:20)\n    at emitOne (events.js:96:13)\n    at TLSSocket.emit (events.js:191:7)\n    at readableAddChunk (_stream_readable.js:178:18)\n    at TLSSocket.Readable.push (_stream_readable.js:136:10)\n    at TLSWrap.onread (net.js:560:20)",
    "bytesParsed": 215,
    "code": "HPE_INVALID_CONSTANT",
    "__error_callsites": [
        {},
        {},
        {},
        {},
        {},
        {}
    ],
    "level": "error",
    "timestamp": "2017-04-18T17:34:09.735Z"
}

From some cursory reading, it seems HPE_INVALID_CONSTANT manifests when the response is malformed. However, these responses are fine until the introduction of the secure agent.

Does anyone have any idea what's going on here or how I can fix it?

Notes: This error occurs on Node v7.9.0 in a Docker container via FROM node:7.9. So far I've only observed this for HEAD requests--there must be some reason for that.

I don't have an exhaustive list of request params, altho I would say in this particular case it seems very unlikely--note these are proxied requests for customer sites so they could be anything. — maxcountryman, Apr 26 '17 at 15:43
I was facing a similar issue. (was using node request library), in my case space in one of the values in request params. Plus i was making the request to http://localhost (nginx) which was then getting redirected by python to https://localhost. The combination of these two were causing the issue for me. One thing i noticed in my case that i had the request body available but the Parse error used to come. I did a small hack to ignore the error, and send the response. I can share the code if you want. PS - using mobile to type pardon any typos or errors — Anuranjit Maindola, Apr 26 '17 at 19:37

score 0 · Answer 1 · answered Apr 27 '17 at 06:29

Just adding the comment here. I was facing a similar issue. (was using node request library), in my case space in one of the values in request params. Plus I was making the request to http://localhost (http)(nginx) which was then getting redirected by python to https://localhost (https). The combination of these two were causing the issue for me. One thing i noticed in my case that i had the request body available but the Parse error used to come. I did a small hack to ignore the error, and send the response.

    let formData = "some data";
    let headers = // some headers;
    headers["content-length"] = formData.length;
    headers["Accept-Encoding"] = "gzip, deflate";
    let gunzip;
    return new Promise(function (resolve, reject) {
        let buffer = [];
        let req = http.request({
            port: 80,
            method: "POST",
            path: url,
            headers: headers
        }, function (res) {
            let resHeaders = res.headers["content-encoding"];
            switch (resHeaders){
                case "gzip":
                    gunzip = zlib.createGunzip();
                    res.pipe(gunzip);
                    gunzip.on("data", function (data) {
                        buffer.push(data.toString());
                    }).on("end", function () {
                        resolve(buffer[0])
                    });
                    break;
                case "deflate":
                    gunzip = zlib.createDeflate();
                    res.pipe(gunzip);
                    gunzip.on("data", function (data) {
                        buffer.push(data.toString());
                    }).on("end", function () {
                        resolve(buffer[0])
                    });
                    break;
                default:
                    res.on("data", function (data) {
                        buffer.push(data.toString())
                    }).on("end", function () {
                        resolve(buffer[0])
                    })
            }
        });
        req.on('error', (e) => {
            // This is the Hack. If there is an error just ignore it.
            logger.debug("Error in getting requests, ", e)
        });
        req.write(formData);
        req.end();
    })
}

Hope it helps. Do share if you figure out the reason. I tried to figure out the reason for a day but couldn't. Since we had this issue in Production so had to do this quick patch.

From what I figured out it is most probably because of HTTPS redirection in the flow.

Why does https.Agent throw a parse error when proxying via node-http-proxy?

1 Answers1