AWStats log format for tomcat access logs which has X-Forwarded-For

Question

What should be the AWStats log format for below tomcat access logs ?

I tried these formats but the external IP addresses are not coming into AWStats reports.

LogFormat="%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot %referer %other %other"

LogFormat="%other %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot %host_proxy"

tomcat valve settings:

pattern="%h %l %{USER_ID}s %t &quot;%r&quot; %s %b &quot;%{Referer}i&quot; &quot;%{User-Agent}i&quot;  &quot;X-Forwarded-For=%{X-Forwarded-For}i&quot;  &quot;JSESSIONID=%{JSESSIONID}c&quot; %D"

Log entry:

127.0.0.1 - - [04/Nov/2013:13:39:55 +0000] "GET / HTTP/1.1" 200 12345 "https://www.google.com/url?some_url" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36"  "X-Forwarded-For=real_ip, proxy_server_internal_ip"  "JSESSIONID=-" 12345

score 2 · Accepted Answer · edited Nov 10 '13 at 05:58

2

$ echo $STR
127.0.0.1 - - [10/Nov/2013:04:14:03 +0000] "GET /XXXX/js/service.js HTTP/1.1" 200 656 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0" "X-Forwarded-For=75.67.17.69, 10.2.0.121" "JSESSIONID=YYYY" 2

$ echo $STR | perl -pe 's/X\-Forwarded\-For=([\d\.]+).*?\"/X-Forwarded-For=$1"/g'
127.0.0.1 - - [10/Nov/2013:04:14:03 +0000] "GET /XXXX/js/service.js HTTP/1.1" 200 656 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0" "X-Forwarded-For=75.67.17.69" "JSESSIONID=YYYY" 2

edited Nov 10 '13 at 05:58

Falcon Momot

25,244
15
63
92

answered Nov 10 '13 at 04:51

Sameer Naik

218
2
8

1

Welcome to server fault! While it's great to give examples, it's nice to explain what is being done so it's useful to future readers as well. Additionally, I've cleaned up the formatting - note that you can mark code segments with a 4-space indent. – Falcon Momot Nov 10 '13 at 05:57
This perl one liner extracts first IP address from X-Forwarded-For field in access log – Sameer Naik Feb 06 '14 at 20:03

score 1 · Answer 2 · answered Nov 14 '13 at 12:04

Thanks for the help.

Did one more trick, awstat wants host_proxy logs as 75.67.17.69, 10.2.0.121, so changed your perl command as below which changes "X-Forwarded-For=75.67.17.69, 10.2.0.121" to "75.67.17.69, 10.2.0.121" without quotes and extra spaces removed.

perl -pe 's/ "X-Forwarded-For=([\d.,\s]+).*?" /$1/g'

After the changes the logs are provided to awstat for processing.

AWStats log format for tomcat access logs which has X-Forwarded-For

2 Answers2