nodejs request-promise url location not showing in headers

Question

I'm currently using request-promise in nodejs to make a request to a website and then return the headers, as I'm trying to get the url (location) of the request incase of redirects. Although the issue I'm having is that the location is not showing up in the headers when I'm logging them after the request.

My code:

const rp = require('request-promise');

const userAgent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36';
const url = //url

var _include_headers = function(body, response, resolveWithFullResponse) {
    return {'headers': response.headers, 'data': body};
  };

const options = {
    uri: url,
    followAllRedirects: true,
    method: 'get',
    gzip: true,
    transform: _include_headers,
    headers: {
        'User-Agent': userAgent
    },
};

const p1 = rp(options).then((response, error, html) => {

    console.log(response.headers);

})

The console logs the headers, but it doesn't show the location in the headers object. Is there anyway I can find the location of the url after making the request and returning the headers?

You're passing `followAllRedirects: true` and the default for `followRedirect` is true. That means that you're telling the `request()` library to HIDE the redirects from you by just following them and giving you only the final result after redirect. If you want to get the redirect status and URL yourself, then you have to tell the `request()` library NOT to follow the redirects automatically so it will return the 3xx status and headers to you. — jfriend00, Dec 16 '17 at 07:24

score 3 · Answer 1 · answered Dec 16 '17 at 07:21

3

What is your request-promise version? You can try this snippet.

var rp = require('request-promise');

    rp({
        uri: 'http://google.com',
        method: 'GET',
        resolveWithFullResponse: true
    })
        .then(function (response) {
            console.dir(response.headers);
        });

answered Dec 16 '17 at 07:21

hoangdv

15,138
4
27
48

perfect, working for me I was not using `resolveWithFullResponse: true` – Ankur Loriya Jun 29 '18 at 12:18

score 3 · Answer 2 · answered Dec 16 '17 at 18:33

When you set followAllRedirects to true, you won't have a location field in your response headers, and the only way to access the final redirected URL is to access the response.request.uri object that contains the final URL in its href property.

To do so, you can modify your transform function to pass an additional finalUrl property that is equal to response.request.uri.href:

const rp = require('request-promise');

const userAgent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36';
const url = "http://www.stackoverflow.com";

const _include_headers = function(body, response, resolveWithFullResponse) {
    return {
        'headers': response.headers, 
        'data': body, 
        'finalUrl': response.request.uri.href // contains final URL
    };
};

const options = {
    uri: url,
    followAllRedirects: true,
    method: 'get',
    gzip: true,
    transform: _include_headers,
    headers: {
        'User-Agent': userAgent
    },
};

const p1 = rp(options).then((response, error, html) => {

    console.log(response.finalUrl);
});

In the example above, http://www.stackoverflow.com is redirected to https://stackoverflow.com, which is what the program prints.

nodejs request-promise url location not showing in headers

2 Answers2