nginx IF statement matches with non-alphanumerical characters

Question

I'm trying to stop some pesky bots by blocking ther user agents in nginx. What is the best way to put multiple user-agents/matches in the same if statement when they have non-alphanumerical characters and need to be encaptulated with quotation marks.

This works:

if ($http_user_agent ~* (python|wget)) {
    return 403;
}

if ($http_user_agent ~* "Opera/9.02 (Windows XP; U; ru)") {
    return 403;
}

if ($http_user_agent ~* "Opera/9.70 (Linux i686 ; U; en) Presto/2.2.1") {
    return 403;
}

But I'd rather something like this(which doesn't in this format):

if ($http_user_agent ~* (python|wget|"Opera/9.02 (Windows XP; U; ru)"|"Opera/9.70 (Linux i686 ; U; en) Presto/2.2.1") ) {
    return 403;
}

Use map directive to group such user agents. Ref: https://nginx.org/r/map — Pothi Kalimuthu, Mar 16 '17 at 06:33
@PothiKalimuthu Oh very interesting, this looks useful thank you for the help. — alexwatever, Mar 16 '17 at 06:52
If it didn't work for some reason, please update the question on what you tried with 'map'. Thanks. — Pothi Kalimuthu, Mar 16 '17 at 11:11
@PothiKalimuthu it worked perfectly, just took some playing around with the regex to get it working right. thanks again, it was a very clean solution. — alexwatever, Mar 16 '17 at 13:03

score 0 · Accepted Answer · answered Mar 16 '17 at 13:06

Thanks to @PothiKalimuthu who suggested using the nginx map directive, which worked perfectly.

My final code looked like this:

# user agent blacklist
map $http_user_agent $ua_bots {
    default                                              0;
    ~*(python|wget)                                      1;
    "~*Opera\/9.02 \(Windows XP; U; ru\)"                2;
    "~*Opera\/9.70 \(Linux i686 ; U; en\) Presto\/2.2.1" 3;
}

# block user agents from map blacklist
if ($ua_bots) {
    return 403;
}

score 0 · Answer 2 · answered Mar 17 '17 at 13:41

I'm glad you've made it up'n running nicely. I wish to add another type of defence and it is somewhat a bit more professional and it will allow user agents who are likely to be blocked now even if they are not really bots. Also, I'm sure you've read te IFISEVIL part from Nginx. :)

I'm using Fail2Ban to protect my webservers via both ssh and http. You just have to make a new jail for them and tell the application to watch the log files and if one line meets the criteria it can ban the IP address, permanently or just for a period of time.

You didn't explain what type of attack are you up against by bots, but there are plenty "out-of-the-box" methods for Fail2Ban to protect a webserver. There is even a specific part written by someone called: Bad bots

[nginx-badbots]
enabled  = true
filter = apache-badbots
action = iptables-multiport[name=BadBots, port="http,https"]
logpath = /var/log/nginx*/*access*.log
bantime = 86400 # 1 day
maxretry = 1

https://snippets.aktagon.com/snippets/554-how-to-secure-an-nginx-server-with-fail2ban

https://www.google.hu/search?q=fail2ban+nginx&oq=fail2ban+nginx&aqs=chrome..69i57.2063j0j4&sourceid=chrome&ie=UTF-8

Great suggestion. I have fail2ban implemented but didn't at the time of the above attack as I'd rebuilt the server that day, due to hardware failure, and hadn't setup fail2ban yet. I realised I didn't know how to do the above with nginx so thought I should find out. Thanks for the advice. — alexwatever, Mar 17 '17 at 20:39
You are welcome. Please share the sollution with us in here. — Bert, Mar 20 '17 at 12:56

nginx IF statement matches with non-alphanumerical characters

2 Answers2