3

From time to time, I see collections of suspect "File not found" errors in my Apache logs, basically using the pattern

File does not exist: /var/www/file, referer: http://my.server.com/file

In human terms: The file was not found, though it referenced here itself. A clear hacking attempt, as that's hardly possible (and the REQUEST_URIs often enough suggest the same). In my eyes a clear case for fail2ban – if I could get backreferences to work here:

failregex = ^%(_apache_error_client)s File does not exist: /var/www(.+), referer: http://.+\1$

(Justin Case: above examples assume the DIRECTORY_ROOT of that webserver being /var/www)

I googled for hours, searched the fail2ban wiki up and down – but nowhere I could find a statement concerning backreferences in its filters. Are they not supported, or did I do it the wrong way? Any hints how to make it work (except from "dirty hacks" like first sending the request to another fake url using , and then catching on that (if anyone is interested, I can elaborate on that approach in an answer), or doing something similar using )?

as an entire log line was requested:

[Fri Nov 08 14:57:28 2013] [error] [client 50.67.234.213] File does not exist: /var/www/text/files.htm++++++++++++++++++++++++++Result:+using+proxy+27.34.142.47:9090;+no+post+sending+forms+are+found;, referer: http://www.myserver.com/text/files.htm++++++++++++++++++++++++++Result:+using+proxy+27.34.142.47:9090;+no+post+sending+forms+are+found;

(sorry, logs were just switched, so this long candidate was the only one left currently; minor adjustments were made for privacy reasons)

Izzy
  • 349
  • 1
  • 5
  • 19
  • Would you post an entire line from the log? – rickhg12hs Nov 09 '13 at 02:05
  • Sure. Just added one to the end of my question. Though it adds no new information: except for the different file path, it's just prefixed with the default Apache Error log information which looks the same with each Apache install ;) – Izzy Nov 09 '13 at 12:44
  • fail2ban uses normal python regexes, so backreferences should work. Do you get a regex compilation error or do things simply not match? What does `fail2ban-regex` say? – Dennis Kaarsemaker Nov 09 '13 at 12:50
  • That's exactly the thing. I've checked the regex, and it works *outside* fail2ban. `fail2ban-regex` simply finds no hits. I tried to narrow it down: `fail2ban-regex` shows matches if I leave the trailing `\1$` off the line; but as soon as I add those 3 chars (or just the `\1`), no more hits. – Izzy Nov 09 '13 at 12:52
  • Just for the records: seems not to be a "hacker", but rather a SEO tool behind those requests (see e.g. [here](http://www.blackhatteam.com/f130/what-is-the-solution-32622.html)). Still, those are not welcome visitors for some of us ;) – Izzy Nov 10 '13 at 13:43

1 Answers1

3

Looks like the backreference numbering is a bit wacky. Try using a Named Group ... something like this:

failregex = ^%(_apache_error_client)s File does not exist: /var/www(?P<snoop_file>.+), referer: http://.+(?P=snoop_file)$
rickhg12hs
  • 394
  • 2
  • 9
  • 2
    Excellent! That did the trick. Be gone, ya SEO crawlers :) And big thanks, rick (+1 & accepted)! – Izzy Nov 11 '13 at 00:28