3

I wonder if anyone else has seen similar phenomena to what a site I help manage is experiencing. For the last two weeks or so, about 10-15 times a day we will get thousands to tens of thousands of requests from a single IP.

These IPs are from all over the world, mostly the US. They all use Firefox revision 52.

Here's an example from our logs:

[06/Apr/2017:11:58:10 -0500] XXX.XXX.XXX.XXX - - "GET / HTTP/1.1" 403 173 "[URL]" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0"

The common part of all of these instances is: "WOW64; rv:52.0) Gecko/20100101 Firefox/52.0". Sometimes the OS is different.

We get about 8-10 requests per second, and then it usually stops (it goes on long after our rate limiter gives them errors). I have seen it make up to 350,000 requests at once.

At first I thought it was from a botnet, since it kept popping up all over. But you'd think they'd do more coordinated attacks.

Then I dug in and realized that sometimes these visitors would browse the site, click an article, or read forum, and then the crazy requests would happen. The specific URIs have nothing in common.

I looked up the IPs of some of these attacks in our database and some are even legitimate community contributors.

My hypothesis is that this is caused by a browser extension that is incompatible in someway with Firefox 52, since it only happens with this revision. The traffic does not seem malicious.

I wonder if anyone is seeing familiar "attacks," or whether anyone has an idea as to the cause.

Erik Westlund
  • 133
  • 4
  • 2
    We see similar behavior on one of our sites with higher traffic. – Yarik Dot Apr 06 '17 at 17:13
  • 2
    You cannot make any conclusions based on the User-Agent shown in the log. The connecting party can send any User-Agent header to the server it wants. So, any malware program can send the User-Agent string of Firefox 52 on its request and the software has nothing to do with Firefox. – Tero Kilkanen Apr 06 '17 at 17:36
  • If a percentage of the same IP addresses request legit pages, my first guess would be the Nework Predictor in Firefox. It has been there since FF 50 or so. In `about:config` search for `network.predictor`. It can be disabled. Also there is an option in haproxy to ignore these probes. Not sure if you have haproxy. – Aaron Apr 06 '17 at 18:05
  • @TeroKilkanen, I know that UAs can be easily spoofed, and of course many bots/malware choose the most common ones to avoid detection. However, in this case: 1. There is legitimate, normal traffic from the same IP and User Agent before the floods with the same user agents. Some of this traffic can be traced back to "normal" postings. 2. The OS in the user agents change, but not the browser version. While possible to spoof this, too, I have never seen that. 3. All the IPs look "normal": residential ISP, past site visitors. 4. It started close to the release of Firefox 52. – Erik Westlund Apr 06 '17 at 20:45

1 Answers1

3

I suspect you are seeing symptomps of this Firefox bug: Cached iframe executes previously loaded and dynamically inserted scripts, makes network calls before "onload" event.

This affects websites in an iframe. Does the website use iframes, or is it a popular-ish site that scammers could have embedded in an iframe?

Here is a comment from the bug:

I am working for AdTech company and after Firefox 52 we noticed increase (about 5x) in ad requests from Firefox browser, but actual number of ads our scripts are able to track left the same.

The fix is out but I would expect a lot of unpached browser for some time.

Looking at the user agent you provided, it doesn't seem to be providing fine grained patch level, so It would be quite hard to block these on server side.

hayalci
  • 3,631
  • 3
  • 27
  • 37
  • This is almost certainly the culprit. We are still getting these types of floods of requests. Without protections, it's still enough sometimes to temporarily bring down a host in our load balancer pool. I've used fail2ban to protect against it by providing 1hr "bans", but that's not an ideal solution: I'd rather not give "too many requests" errors to legitimate readers, but the problem is rare enough that it seems like an OK temporary compromise to avoid unhealthy host servers. – Erik Westlund Apr 19 '17 at 14:32