I asked this question last week, and followed the debugging advice and now have additional information.
I have an odd problem-- on a high traffic website (millions of visitors a month), every day we get about 20 or so situations where one host begins incessantly requesting the same page, over and over-- multiple times per second, for any length of time from a few minutes to all day.
The attack is apparently not malicious, as I've backtraced the IP address and matched them to some of our registered users, whom I've interviewed. They say that when this happens, their computer becomes slow, but it is otherwise usable. It doesn't happen on every page load, but rather sporadically.
The log hits have the following characteristic:
- They start 'normal'-- first page load actually accesses all the page's resources (images, etc), as well as the .php
- Then the host begins requesting JUST the php page, without the resources incessantly, usually one per second (but sometimes faster and sometimes a few seconds slower)
- The remote browser is always Firefox 3.x (we've seen as high as 3.5.3 and as low as 3.0.2)
- Subsequent hits have no referrer, even though the first page request had one
- This goes on as long as the visitor is on that particular page, and the next page load is often normal
- The user is often oblivious to what's happening-- their page does not reload, so whatever is happening is going on in the background for them, and I believe Firefox is simply throwing the information away
- This happens on both static (e.g., contact) and dynamic pages (e.g., mailbox)
- The IP distribution of the affected hosts has no similarities I can discern (e.g., they are not all behind a corporate firewall)
- We thought a bad javascript on the page could cause it, but disabling the javascript entirely did not affect the problem at all
We are at wit's end with what to do with this. A simple DoS filter is not appropriate-- we have that and the threshold to trigger it is much higher than a single page request (without related images, css, etc.) a second. We also installed mod_evasive, but that's not catching these as we have a multiple server system and that operates on a per-child basis.
The stack is LAMP, Redhat install, PHP 5.2, Apache 2.2.3, with an NGINX box operating as a software load balancer in front of multiple web servers and database back ends.
In the absence of good ideas, we resorted to writing our own dummy filter in memcached that stores a key of IP+URI in memcached IFF the user is Firefox 3.x and the referrer is blank, and increments each page request. Once it crosses a certain threshold in a certain period of time, we 403 further requests.
I do NOT think this is the appropriate place in the networking stack to handle this issue, however. Hoping someone else has seen these and can help us find the pattern.
Thank you for anything you can contribute!
