To clarify, I do not wish to limit the number of simultaneous connections, nor do I want to limit the number of HTTP requests. I only want to limit the number of NEW connections per IP.
I want to do this because most web crawlers do not have keep-alive functionality and thus they open a new connection for every request.
I vaguely remember reading about a mod that could do this, but I can't remember the name. Hopefully, someone here can help me out.
I wouldn't do it in apache.. I'd do it at network layer with iptables.
iptables -A INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --set
iptables -A INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --update --seconds 86400 --hitcount 100 -j REJECT
Change 86400 to the number of seconds you want to keep the block for (86400 is 1 day), and 100, is the hit count, how many you're prepared to allow per IP.
You can also change -j REJECT to -j DROP, which defines the packet behaviour when the condition is met. DROP seamlessly drops packets, and REJECT returns a "port unreachable" or similar error.
That said, there was a mod_throttle that would do something similar, but I can't seem to find much information about it. I think it feels neater to do this kind of thing at the network/kernel level, rather than in Apache. Apache is good at serving requests. Let it do what it does best, and don't burden it with having to track connections too.
you could tune the iptables rule so that requests beyond the limit are redirected to another port (e.g. TCP 8080) where another instance of Apache serves a static page with your message to users
I think you are looking for mod_qos
