mod_rewrite Fast and low resource way to block the request

Question

I use following rule to block requests based on some User Agents using modRewrite

RewriteCond %{HTTP_USER_AGENT} ^.*(SCspider|PetalBot|ZyBorg).*$ [NC]
RewriteRule ^.*$ - [F,L]

OR

RewriteCond %{HTTP_USER_AGENT} ^.*(SCspider|PetalBot|ZyBorg).*$ [NC]
RewriteRule .* - [F,L]

But I need to know if there is any better way (quick & fast) and less resource consumption to block/drop the requests
appreciate for any help

Which kind of user agents you want to block? could you please post some examples then we could help more, thank you. — RavinderSingh13, May 11 '21 at 15:30
@RavinderSingh13 I just need the the best part of blocking , not user agent identifing — Farhad Sakhaei, May 11 '21 at 15:42
I understand but which urls or bots you want to block? Or everything(any request) you want to block? — RavinderSingh13, May 11 '21 at 15:43

score 3 · Accepted Answer · answered May 11 '21 at 15:50

3

With your shown samples try following. Following should be faster than your tried one.

RewriteCond %{HTTP_USER_AGENT} (?:SCspider|PetalBot|ZyBorg) [NC]
RewriteRule ^ - [F,L]

Suggested improvements:

We need to use .* in condition part since anything that contains these strings should be blocked, you could change it to \b(?:SCspider|PetalBot|ZyBorg)\b to avoid partial matches too(just in case).
Then you are creating capturing group in condition which is NOT required since its NOT being used anywhere later on.
In RewriteRule portion also we need not to use .* we could simply use ^ to avoid matching everything, since we are anyway doing that match in Condition part.

answered May 11 '21 at 15:50

RavinderSingh13

130,504
14
57
93

Do you thin that RewriteRule ^ - [R=204] is faster and low resource base or RewriteRule ^ - [F,L] ? Thank you for explanations – Farhad Sakhaei May 11 '21 at 15:58
@FarhadSakhaei, they are different, because `R=200` will give 200 ok result but `F` is forbidding the urls. – RavinderSingh13 May 11 '21 at 15:59
Sorry I wrote wrong status code, it's 204 : no content – Farhad Sakhaei May 11 '21 at 16:00
I need something like drop in modsecurity rather than block or deny – Farhad Sakhaei May 11 '21 at 16:01
@FarhadSakhaei, oh ok no issues. IMHO `F` should be more faster than `R` because that's kind of external redirection but haven't tested it. – RavinderSingh13 May 11 '21 at 16:01
@FarhadSakhaei, yeah, these rules should be Good, please test them out once and let me know if these help you. – RavinderSingh13 May 11 '21 at 16:02
I read somewhere which R = 204 is fast, But I'm not sure, just asked for that, I don't know how to benchmark something like that – Farhad Sakhaei May 11 '21 at 16:05
@FarhadSakhaei, to be honest, if we correct and use Good regex(like I fixed most of it) then it shouldn't be too much difference IMHO, may be few milli seconds here and there, but if you keep these rules top of your htaccess file(no exceptions considering here) then these should be faster IMHO. Let me know your views on same, thank you. – RavinderSingh13 May 11 '21 at 16:07
@FarhadSakhaei, Also why I am saying using correct regex will help, since `.*` and creating capturing group will definitely take time because they will use memory to match and keep values safe in memory respectively. – RavinderSingh13 May 11 '21 at 16:16
Thank you for explanations, How can I chat with you? – Farhad Sakhaei May 11 '21 at 16:53
@FarhadSakhaei, Is it apart from question or related to question, if related to question we could chat here. – RavinderSingh13 May 11 '21 at 16:54
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/232244/discussion-between-farhad-sakhaei-and-ravindersingh13). – Farhad Sakhaei May 11 '21 at 17:03
1

Thank you it helped me, great regular expression explanation – Farhad Sakhaei May 11 '21 at 17:30
@FarhadSakhaei, your welcome, cheers and happy learning. – RavinderSingh13 May 11 '21 at 17:32

mod_rewrite Fast and low resource way to block the request

1 Answers1