Some mod_security rules not being applied

Question

We've been seeing a lot of referral spam to one of our servers so I decided to add some custom modsecurity rules to try and stop at least some of them.

I've added several rules, for instance :

SecRule REQUEST_HEADERS:User-Agent "/\byourekillingme.org\b/" \
    "phase:1,log,deny,status:503,msg:'Referer spam1',id:101"

SecRule REQUEST_HEADERS:User-Agent "/\bahrefs.com/robot\b/" \
    "phase:1,log,deny,status:503,msg:'Referer spam2',id:102"

SecRule REQUEST_HEADERS:User-Agent "/\bsemrush.com/bot\b/" \
    "phase:1,log,deny,status:503,msg:'Referer spam6',id:106"

but as you can see from the log output, while the ahrefs rule (id 102 above) is being applied, the others aren't (there are others, but ahrefs is the only one working) :

107.180.120.23 - - [23/Nov/2017:11:08:00 +0000] "GET /tri-levlen-28-side-effects-3f1 HTTP/1.1" 200 50965 "http://www.mydomain.co.uk/tri-levlen-28-side-effects-3f1#elephant" "WordPress/4.9; http://yourekillingme.org"

51.255.65.42 - - [23/Nov/2017:10:40:51 +0000] "GET /pink-viagra-price-52c HTTP/1.1" 503 315459 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"

46.229.168.73 - - [23/Nov/2017:11:07:50 +0000] "GET /viagra-for-sale-online-cheap-52c HTTP/1.1" 200 51060 "-" "Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html)"

I've also checked /usr/local/apache/logs/modsec_audit.log to confirm it is my rule that's causing the 503 and not another rule somewhere else, and that just shows lots of entries for :

Message: Access denied with code 503 (phase 1). Pattern match "/\\bahrefs.com/robot\\b/" at REQUEST_HEADERS:User-Agent. [file "/usr/local/apache/conf/modsec2.myrules.conf"] [line "8"] [id "102"] [msg "Referer spam2"]

indicating it is my modsec rule stopping it.

Can anyone see why my other rules aren't being applied?

score 2 · Accepted Answer · answered Nov 23 '17 at 12:37

It's the slashes. Regular expressions in ModSecurity are not bounded by slashes, so those are interpreted as a literal part of the regular expression.

All of your expressions are matching URLs, so the first slash matches the final slash in the http:// part of the URL, but only the AHrefs URL has a trailing slash so that's the only one that ends up matching these regular expressions.

I'm not sure the \b is doing anything useful either. It seems to match these three either at the . or the / or the end of the string. If it is actually understood by ModSecurity it may cause future expressions for other bots to fail to match when you expect them to match.

Thanks, that's fixed it! Thought I'd tried with just the text string, but that must have been before I fixed a previous issue. — Keith Langmead, Nov 23 '17 at 13:05

Some mod_security rules not being applied

1 Answers1