Nginx rate limiting on unique IPs

Question

we've been dealing with constant attacks on our authentication url, we're talking millions of requests per day, my guess is they are trying to brute force passwords.

Whenever we would block the IP with the server firewall, few seconds later the attacks would start again from a different IP.

we ended up implementing a combination of throttling through rack-attack plus custom code to dynamically block the IPs in the firewall. But as we improved our software's security, so did the attackers, and now we are seeing every request they make is done from a different IP, one call per IP, still several per seconds, not as many but still an issue.

Now i'm trying to figure out what else can i do to prevent this, we tried recaptcha but quickly ran out of the monthly quota and then nobody can login.

I'm looking into Nginx rate limiter but from what I can see it also uses the IP, considering they now rotate IPs for each request, is there a way that this would work?

Any other suggestions on how to handle this, maybe one of you went through the same thing?

Stack: Nginx and Rails 4, Ubuntu 16.

Answer 1

With rate limiting you filtered out some of the high volume password sprayers, which helps. However, IP based rate limiting alone does not work when the attacker has many IP addresses, quickly rotates through them, and limits requests to a human rate.

we tried recaptcha but quickly ran out of the monthly quota and then nobody can login.

Pay for your CAPTCHA service and observe how it works at scale for the month. If you do not like the pricing, look at alternatives. hCaptcha, for example, has a basic free tier.

More fundamentally, replace passwords with something better, maybe hardware security keys or biometrics. Or at least supplement passwords with additional auth factors. Standards exist for doing this, namely FIDO2. And for non user interactive credentials, ssh keys, x509 certificates, or generated API keys are also better than user selected passwords.