Somebody eating all our bandwidth, what should I do?

Question

OK, this is frustrating, my site got thousands of page views/connections in the last 3 days and finally we ran out of monthly bandwidth. We bought more bandwidth and closed the site for maintenance. We checked the logs and found a responsible IP and banned it, but when we opened the site the attack continued. This time it was from multiple IPs from different countries, they visited different pages of our website thousands of times.

What should we do?

EDIT This might be important: The robot or hacker or whatever seems to stick to one webpage and visit it over and over again (like our forum membership page) but when we restrict that page's permissions it just goes and finds other places. Weird.

score 5 · Accepted Answer · edited May 16 '11 at 19:23

5

Don't block the robot completely, but slow it down with QoS rules (in linux, check out the "tc" command). Also, read this. This is how the people running this very website solved the problems using Haproxy.

edited May 16 '11 at 19:23

Holocryptic

5,665
2
29
37

answered May 16 '11 at 19:19

Julien Vehent

3,017
19
26

Thanks, We are using a normal linux host, can we do that without directly having access to the servers or should we contact our providers? – Auxiliary May 16 '11 at 19:22
1

If you don't have command line access, you won't be able to install haproxy. So you can't. For QoS, you need root access since it affect the processing of networks packets in the kernel. – Julien Vehent May 16 '11 at 20:06

score 2 · Answer 2 · answered May 16 '11 at 19:36

I would like to suggest PHREL. Very small and simple tool that makes dynamically blocking abusive IPs a breeze. I deployed it on some caching nameservers with immediate results. Test it out a bit and figure out what your thresholds should be and let it rip!

http://www.digitalgenesis.com/software/phrel/

score 1 · Answer 3 · answered May 16 '11 at 19:24

Playing whack-a-mole is usually a fruitless effort, but it may be the only option here.

You could possibly automate the identification of your bandwidth hogs. I would look at monitoring the web logs and identifying anomalous behavior such as the same IP looking at the same page more than once a second over a defined period of time, or some other similar metric. You don't mention which platform this is, but it would be a bit easier to implement in Linux than Windows.

Thanks, it's a linux host with the CPanel, we've got wordpress and MyBB installed on it. — Auxiliary, May 16 '11 at 19:33

Somebody eating all our bandwidth, what should I do?

3 Answers3