How to capture http messages from Request Node library with Fiddler

Question

Regular client initiated requests to the node server are captured fine in Fiddler. However, requests sent from node to a web service are not captured. It did not help to pass in config for proxy (127.0.0.1:8888) to the request method. How can I route the request messages through Fiddler?

var http = require('http');
var request = require('request');

request.get(webserviceURL, { "auth" : {"user": "user", "pass" = "pass", sendImmediately: true },
"proxy" : { "host" : "127.0.0.1", "port" : 8888 }},
function (error, response) { console.log( "response received" );
});

Request repo: https://github.com/mikeal/request

Setting the proxy on the request object *should* be all you need to do, but I don't see a "proxy" property in the docs? http://nodejs.org/api/http.html#http_http_request_options_callback — EricLaw, Jun 29 '13 at 18:38
I'm using the third-party request package. Link in description. — mupersan82, Jun 29 '13 at 18:41
Poking around in the source, it seems to imply that the syntax would be "hostname" rather than "host"? Or you could define proxy as the string "http://127.0.0.1:8888" rather than as an object? — EricLaw, Jun 29 '13 at 18:44
Changing "host" to "hostname" did not help. Can you get it working? — mupersan82, Jun 29 '13 at 18:52

idolize · Accepted Answer · 2014-12-08T00:29:13.090

26

I just tried to do this myself (using Fiddler and the request library from npm). Here's how I got it working:

process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0'; // Ignore 'UNABLE_TO_VERIFY_LEAF_SIGNATURE' authorization error

// Issue the request
request(
{
    method: "GET",
    uri: "https://secure.somewebsite.com/",
    proxy: "http://127.0.0.1:8888" // Note the fully-qualified path to Fiddler proxy. No "https" is required, even for https connections to outside.
},
function(err, response, body) {
    console.log("done");
});

This is with Fiddler2 using the default port and proxy options (and no proxy authentication).

edited Dec 08 '14 at 00:29

answered Apr 10 '14 at 18:25

idolize

6,455
4
24
34

2

Thanks a lot. the setting NODE_TLS_REJECT_UNAUTHORIZED really helps. Without it, https request won't work for me. Just curious, what does this environment variable mean? – Habibi Dec 06 '14 at 12:38
I believe it allows Node to use self-signed SSL certificates, which would normally be a security risk, but in this case it's what you want :) – idolize Oct 12 '15 at 22:25

score 5 · Answer 2 · answered Jun 29 '13 at 18:59

5

Fiddler works by setting your "Internet Options" (from start menu) "Connections" > "LAN Settings" > "Proxy Server" to its port, thus making all HTTP traffic (clients which obey this setting) go through it.

You should point your node.js client lib to use a proxy, the settings are written in that options dialog after you start Fiddler.

answered Jun 29 '13 at 18:59

Poni

11,061
25
80
121

Did you edit your question or did I get a REAL blackout and missed that part in your code example? LOL – Poni Jun 30 '13 at 19:27
1

Although proxy is implicitly mentioned in the first post, this one solved the problem for me by drawing my attention more explicitly. Thanks @poni. – sfuqua Oct 14 '15 at 21:11

score 4 · Answer 3 · answered Apr 08 '14 at 12:21

4

The proxy option should be a full url, like this:

proxy : "http://127.0.0.1:8888"

answered Apr 08 '14 at 12:21

Fernando Tiberti

1,272
2
10
18

score 3 · Answer 4 · answered Apr 10 '17 at 11:31

3

To do this on an ad-hoc basis, without changing your code, you can use environment variables.

Request respects:

HTTP_PROXY
HTTPS_PROXY
NO_PROXY

So, to proxy just set these in your console before running your process.

For example, to setup http and https proxy use:

set HTTP_PROXY="http://127.0.0.1:8888"
set HTTPS_PROXY="http://127.0.0.1:8888"
set NODE_TLS_REJECT_UNAUTHORIZED=0

The latter line stops issues with SSL through the fiddler proxy.

answered Apr 10 '17 at 11:31

Jon Egerton

40,401
11
97
129

1

For me, on Windows 10, the first 2 lines did not work. It works without the quotes.. – RaelB Jan 02 '20 at 14:04

Doug Coburn · Answer 5 · 2018-10-19T21:05:35.117

I've been wanting the same... an equivalent of the Network tab in chrome DevTools, only for Nodejs. Unfortunately, it doesn't appear as though one exists. I don't have Fiddler on macos, so this is how I went about stubbing the require('http') methods to log and pass though. Leaving this here in case I need it again or someone else finds it helpful. You can turn it on by attaching a debugger and require('filename')() the file containing this script.

module.exports = () => {
    const http = require('http');
    http._request = http.request;

    global.DO_LOG_AJAX = true;
    const log = str => {
        if (global.DO_LOG_AJAX) {
            console.debug(str);
        }
    };

    const flushLog = (requestLines, responseLines) => {
        if (global.DO_LOG_AJAX) {
            log([
                '----------------Begin Request-----------------------------------',
                ...requestLines,
                '----------------End Request / Begin Response--------------------',
                ...responseLines,
                '----------------End Reponse-------------------------------------',
            ].join('\n'));
        }
    };

    let write;
    let end;
    http.request = (...requestParams) => {
        const req = http._request(...requestParams);
        const { method, path, headers, host, port } = requestParams[0];
        const requestLogLines = [];
        requestLogLines.push(`${method} ${path}`);
        requestLogLines.push(`Host: ${host}:${port}`);
        for (const header of Object.keys(headers)) {
            requestLogLines.push(`${header}: ${headers[header]}`);
        }
        write = write || req.write;
        end = end || req.end;

        req.on('error', err => {
            log({ err });
        });

        req._write = write;
        req._end = end;
        const requestBody = [];
        req.write = (...writeParams) => {
            requestBody.push(writeParams[0].toString());
            return req._write(...writeParams);
        };
        req.end = (...endParams) => {
            if (endParams[0]) {
                requestBody.push(endParams[0].toString());
            }
            requestLogLines.push('');
            requestLogLines.push(requestBody.join(''));
            return req._end(...endParams);
        };

        const responseLogLines = [];
        req.once('response', response => {
            const responseBody = [];
            responseLogLines.push(`${response.statusCode} ${response.statusMessage}`);
            for (const header of Object.keys(response.headers)) {
                responseLogLines.push(`${header}: ${response.headers[header]}`);
            }
            const onData = chunk => {
                responseBody.push(chunk.toString());
            };
            const onClose = err => {
                responseLogLines.push('');
                responseLogLines.push(responseBody.join(''));
                responseLogLines.push('');
                responseLogLines.push(`--- ERROR --- ${err.toString()}`);
                flushLog(requestLogLines, responseLogLines);
                req.removeListener('data', onData);
            };
            const onEnd = () => {
                responseLogLines.push('');
                responseLogLines.push(responseBody.join(''));
                flushLog(requestLogLines, responseLogLines);
                req.removeListener('data', onData);
            };
            response.on('data', onData);
            response.once('close', onClose);
            response.once('end', onEnd);
        });

        return req;
    };
};

How to capture http messages from Request Node library with Fiddler

5 Answers5

Linked