0

Search keywords/phrases are not appearing in the referrer link in the Apache log.
Is there an additional Logformat or other settings needed, what am I missing? Running Apache 2.4 on Ubuntu

Here's the log format in the apache2.conf (default) :

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
CustomLog /var/log/apache2/access.log combined

To test the referrer capture, I entered a keyword to search on bing.com and google.com, the web site was found and listed, and I clicked the links. I expect to see the search keyword in the referrer URL. In the access.log I only see the referrer but search keywords or parameters are missing in the URL:

....
"GET / HTTP/1.1" 200 11868 "https://www.bing.com/" "Mozilla/5.0 (Linux; Android 9; Mi 9 SE) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.210 Mobile Safari/537.36"
"GET /solutions/ HTTP/1.1" 200 7569 "http://www.google.com/" "Mozilla/5.0 (Linux; Android 9; Mi 9 SE) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.210 Mobile Safari/537.36"
....
can.do
  • 101
  • 2

1 Answers1

2

This has nothing to do with Apache, and there is nothing you can change to get the search keywords.

Browsers are enhancing their privacy models by reducing information sent via the Referer header. This means that the default policy is to send only domain root URL in the Referer header on cross-domain requests.

The change has been implemented in Chrome 85.

https://developers.google.com/web/updates/2020/07/referrer-policy-new-chrome-default explains the changes in more detail.

Tero Kilkanen
  • 36,796
  • 3
  • 41
  • 63
  • I see ... thanks. I can understand the security aspect but passing this as a privacy concern doesn't seem right when your zip code and other info is available as you browse. I should be able to know how users reached my site, so I can improve and serve better; If I'm selling lemonade I want to fine tune and try to exclude people looking for orange juice. At least secure webs should be able to use it, it's a discussion topic I suppose, but I'm disappointed to find out about it. – can.do May 25 '21 at 21:11
  • You can use https://search.google.com/search-console/about to see what keywords users are searching when they land on your site. – Tero Kilkanen May 26 '21 at 06:15
  • Thank you I'll take a look. I use analytics.google.com and that doesn't seem to display keywords either, but I'll give this a try. – can.do May 27 '21 at 13:45