2

When setting the referer header on an HttpWebRequest I'm seeing two different behaviors. On some sites the referer header will remain as each redirect is followed while on others the referer header is dropped after the first request. What would cause this behavior and is there a way to control it?

Sample (this appears to be HTTPS specific but am not sure why)

In this request the header will be dropped on the redirect.

        HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://mail.google.com/mail/");
        request.Referer = "http://www.google.com";
        HttpWebResponse response = (HttpWebResponse)request.GetResponse();

In this request the header remains on each redirect followed. The difference in the second request is http vs https. 

        HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://mail.google.com/mail/");
        request.Referer = "http://www.google.com";
        HttpWebResponse response = (HttpWebResponse)request.GetResponse();
tribus
  • 1,110
  • 1
  • 9
  • 27
  • Can you please clarify "referer header is dropped" statement? (preferably with code sample)... – Alexei Levenkov Apr 03 '15 at 00:41
  • Using the code above to make a request. When I request a page from site one that returns a 302 response, the subsequent request when the redirect is followed still has the referer header set. When I use the same code to request a page from site two that returns a 302 response, the subsequent request when the redirect is followed no longer contains the referer header. The referer header is only there for the original request but is dropped when following the redirect. This is what I'm seeing in Fiddler. – tribus Apr 03 '15 at 01:00
  • I've added an example to clarify. – tribus Apr 03 '15 at 01:14
  • Browsers drop referrer on http/https transition, so it seems to be reasonable for .Net to do the same... Not sure if .Net behavior is documented somewhere. If you have to keep it - perform 302 manually by disabling "follow redirects" for requests. – Alexei Levenkov Apr 03 '15 at 04:12
  • I agree. However I am seeing exactly the opposite behavior. On the first request the redirect goes from https to https and the header is dropped. On the second, when going from http to https the header remains through all redirects. – tribus Apr 03 '15 at 11:30
  • 1
    You can check in [sources](http://referencesource.microsoft.com/#System/net/System/Net/HttpWebRequest.cs,f2ab2d685cb26f13) what actually happening, but if you need very particular behavior I'd stick with manual redirect. – Alexei Levenkov Apr 03 '15 at 16:31

1 Answers1

1

The referer header is stripped when coming from any HTTPS site regardless of where it is going:

http://referencesource.microsoft.com/#System/net/System/Net/HttpWebRequest.cs,5782

tribus
  • 1,110
  • 1
  • 9
  • 27