30

I'm looking into rate-limiting using nginx's HttpLimitReqModule. However, requests are all coming from the same IP (a loadbalancer), with the real IP address in the headers.

Is there a way to have nginx rate-limit based on the ip in the X-Forwarded-For header instead of the ip of the source?

John Brodie
  • 403
  • 1
  • 4
  • 6

3 Answers3

34

Yes, typical rate-limiting configuration definition string looks like:

 limit_req_zone  $binary_remote_addr zone=zone:16m rate=1r/s;

where $binary_remote_addr is the unique key for limiter. You should try changing it to $http_x_forwarded_for variable which gets the value of X-Forwarded-For header. Although this will increase memory consumption because $binary_remote_addr is using compressed binary format for storing IP addresses and $http_x_forwarded_for is not.

 limit_req_zone  $http_x_forwarded_for zone=zone:16m rate=1r/s;
markdsievers
  • 103
  • 4
Andrei Mikhaltsov
  • 3,027
  • 1
  • 23
  • 31
  • I just came to the same conclusion, and in a quick test it works fine. Thanks for pointing out the increased memory usage. – John Brodie Mar 13 '13 at 14:59
  • 4
    Beware there may be serious security concerns to this: http://blog.ircmaxell.com/2012/11/anatomy-of-attack-how-i-hacked.html – ircmaxell Apr 08 '15 at 19:34
  • Note that the information in that blog post regarding symfony was addresses with the following: http://symfony.com/doc/current/components/http_foundation/trusting_proxies.html – calumbrodie Sep 21 '15 at 13:44
  • 8
    If you use realip module, `$binary_remote_addr` variable gets set correctly. – Cenk Alti Jan 10 '17 at 17:35
  • What is 16m in this context? – Esqarrouth Mar 18 '21 at 11:01
  • @Esqarrouth The size of shared memory, namely “_zone_,” for that. 16 MiB. – Константин Ван May 31 '21 at 21:07
  • **Do not trust HTTP headers. It’s not something you can’t set to anything you want.** – Константин Ван May 31 '21 at 21:09
  • 2
    @КонстантинВан you can trust request headers if you know for sure they are always set or cleaned by a trusted reverse proxy/load balancer. Reject all non-proxied connections with a firewall or non-public network, or use a whitelist (on the original `$remote_addr`) for header usage like the RealIP module does. – Lukas Jul 11 '21 at 15:32
6

The limit_req_zone directive defines the variable to be used as key for request grouping.
Usually, the $binary_remote_addr is used rather than $remote_addr because it is smaller and saves space.

Maybe you alternatively want to use the ngx_http_realip_module.
This will rewrite the remote address variables to the address provided in a custom header and will also make logging and other variable usage easier.

aland
  • 172
  • 1
  • 7
Lukas
  • 1,004
  • 6
  • 14
  • 1
    +1 for RealIP module. When using this module, `$binary_remote_addr` and `$remote_addr` are set to the value of your configured header, typically `X-Forwarded-For` - so your standard variables are now the "real client IP address". Run `nginx -V` to see if NGINX was built with `--with-http_realip`. Then config is as simple as: `set_real_ip_from 10.0.0.0/8;` `real_ip_header X-Forwarded-For;` , where the CIDR range is that of your upstream load balancer which is setting the `X-Forwarder-For` header. – markdsievers Jul 09 '19 at 21:33
0

Just as an example and demonstration of setting up real_ip using the header.

  • We precise a header to take the ip from

  • The client ip will be then overridden with that value

  • Then we can keep using $binary_remote_addr

    • not always x-forwarded-for is used.

  • That method can help too with logs (default format will work as $remote_addr now it's set to the overridden value)

  • Nginx Module to use: ngx_http_realip_module Example of setting that up with Cloudflare (more than one server source, or ip range)

# Ipv4
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 104.16.0.0/13;
set_real_ip_from 104.24.0.0/14;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 131.0.72.0/22;

# Ipv6
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2405:8100::/32;
set_real_ip_from 2a06:98c0::/29;
set_real_ip_from 2c0f:f248::/32;

real_ip_header CF-Connecting-IP;
  • you can have that in a file. ex: cloudflare_ip.conf and add it to conf.d

You can see the usage of

  • set_real_ip_from

    • only the requests coming from those ip's, the overriding will be applied to

  • real_ip_header

  • The header containing the client ip connecting through Cloudflare is CF-Connecting-IP

  • Cloudflare guide

And for the rate limit

limit_req_zone $binary_remote_addr zone=limit_ip_10rs:10m rate=10r/s;
  • We keep using $binary_remote_addr
Mohamed Allal
  • 111
  • 5