5

I have a service with a large share of requests with an empty value for HTTP_REFERER. I'd like to interpret this correctly and wonder about the most common reasons for that.

I understand that HTTP_REFERER is an optional header field, but most browsers with default setting seem to send them.

Common reasons I have found so far:

  • proxies
  • robots
  • JavaScript links (All of them? Is this browser dependant?)
  • request from bookmarks or as browser startup page
  • user entered URL manually
  • Flash links
  • link from a different app like email client
  • browser settings or privacy browser add-ons
  • some personal firewalls filter referrers
  • no referrer is sent by most browsers if the redirect happens via semi-official Refresh http header
  • referrer fakers like this

What's missing|irrelevant|wrong?

Is it possible to put percentages behind these items? Or at maybe sort the list and point out the proportions?

the.duckman
  • 6,376
  • 3
  • 23
  • 21
  • This is probably getting more common because google chrome's default home page (which is wonderful) is a jump-off to frequently used web pages, yet it isn't a web page itself. – Zack Marrapese Jun 09 '09 at 21:24

4 Answers4

3

A percentage will depend on what your website is and why people may want to fake their referrer .. Also some people just crack open a new-tab without a homepage. Or land via something other than the browser (such as an addon or chat link, whatever).

If your functionality relies on the referrer use a cookie or rethink the design. Because you can't rely on it.

Aiden Bell
  • 28,212
  • 4
  • 75
  • 119
0

In some cases 301 redirects are the cause of losing referral information.

Jason Plank
  • 2,336
  • 5
  • 31
  • 40
Evgeny
  • 6,533
  • 5
  • 58
  • 64
  • In what cases? It doesn't seem to be the case on my server. I'm redirecting non-www to www and http to https, and both 301s seem to retain referral information, if it is present. –  Jun 29 '22 at 08:50
  • When a page redirects to your page, it is reasonable to see that page as the referrer. When that page received several redirects already, the original page that actually sent the referral is lost or at least the value in `HTTP_REFERRER` is not the actual page that sent the traffic to your website. At least that is what I think I meant 12 years ago when I wrote this. – Evgeny Jun 30 '22 at 09:23
0

Basically, all page requests that does not involve user clicking on a link on a webpage.

David
  • 5,356
  • 2
  • 26
  • 39
  • 2
    Well, that's not all there is to it. Some software (google suggests Norton Internet Security for example) also masks http-referrer headers. – ChristopheD Jun 09 '09 at 20:40
0

All depends, and we don't have enough information to say which of the causes is most likely. I'd say robots, but you have to analyse the data (assuming you have server logs) and interpret it. I have no idea how popular your site is or what is its purpose, so robots may not be the number one reason.

Armandas
  • 2,276
  • 1
  • 22
  • 27