1

I have an nginx reverse proxy serving multiple small web services. Each of the servers has different domain names, and are individually protected with SSL using Certbot. The installation for these was pretty standard as provided by Ubuntu 20.04.

I have a default server block to catch requests and return a 444 where the hostname does not match one of my server names. However about 3-5 times per day, I have a request getting through to my first server (happens to be Django), which then throws the "Not in ALLOWED_HOSTS" message. Since this is the first server block, I'm assuming something in my ruleset doesn't match any of the blocks and the request is sent upstream to serverA

Since the failure is rare, and in order to simulate this HOST_NAME spoofing attack, I have tried to use curl as well as using netcat with raw text files to try and mimic this situation, but I am not able to get past my nginx, i.e. I get a 444 back as expected.

Can you help me 1) simulate an attack with the right tools and 2) Help identify how to fix it? I'm assuming since this is reaching my server, it is coming over https?

My sanitized sudo nginx -T, and an example of an attack are shown below.

ubuntu@ip-A.B.C.D:/etc/nginx/conf.d$ sudo nginx -T
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 768;
}

http {
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        # SSL Settings
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        # Logging Settings
        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        # Gzip Settings
        gzip on;

        # Virtual Host Configs
        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}

# configuration file /etc/nginx/modules-enabled/50-mod-http-image-filter.conf:
load_module modules/ngx_http_image_filter_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-xslt-filter.conf:
load_module modules/ngx_http_xslt_filter_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-mail.conf:
load_module modules/ngx_mail_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-stream.conf:
load_module modules/ngx_stream_module.so;

# configuration file /etc/nginx/mime.types:
types {
    text/html                             html htm shtml;
    text/css                              css;
    # Many more here.. removed to shorten list
    video/x-msvideo                       avi;
}

# configuration file /etc/nginx/conf.d/serverA.conf:
upstream serverA {
  server 127.0.0.1:8000;
  keepalive 256;
}

server {
  server_name serverA.com www.serverA.com;
  client_max_body_size 10M;

  location / {
    proxy_pass  http://serverA;
    proxy_http_version 1.1;
    proxy_set_header        Host            $host;
    proxy_set_header        X-Real-IP       $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        Upgrade         $http_upgrade;
    proxy_set_header        Connection      "Upgrade";
    }

    listen 443 ssl; # managed by Certbot
    ssl_certificate ...; # managed by Certbot
    ssl_certificate_key ...; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}



server {
    if ($host = serverA.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    if ($host = www.serverA.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    listen 80;
    server_name serverA.com www.serverA.com;
    return 404; # managed by Certbot


}

# configuration file /etc/letsencrypt/options-ssl-nginx.conf:
# This file contains important security parameters. If you modify this file
# manually, Certbot will be unable to automatically provide future security
# updates. Instead, Certbot will print and log an error message with a path to
# the up-to-date file that you will need to refer to when manually updating
# this file.

ssl_session_cache shared:le_nginx_SSL:10m;
ssl_session_timeout 1440m;
ssl_session_tickets off;

ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers off;

ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA";

# configuration file /etc/nginx/conf.d/serverB.conf:
upstream serverB {
  server 127.0.0.1:8002;
  keepalive 256;
}
server {
  server_name serverB.com fsn.serverB.com www.serverB.com;
  client_max_body_size 10M;

  location / {
    proxy_pass  http://serverB;
    ... as above ...
    }


    listen 443 ssl; # managed by Certbot
    ... as above ...

}
server {
    if ($host = serverB.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

   if ($host = www.serverB.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

   if ($host = fsn.serverB.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

  server_name serverB.com fsn.serverB.com www.serverB.com;
  listen 80;
  return 404; # managed by Certbot
}

# Another similar serverC, serverD etc.

# Default server configuration
#
server {
        listen 80 default_server;
        listen [::]:80 default_server;

#       server_name "";
        return 444;
}

Request data from a request that successfully gets past nginx to reach serverA (Django), where it throws an error: (Note that the path will 404, and HTTP_HOST headers are not my server names. More often, the HTTP_HOST comes in with my static IP address as well.

Exception Type: DisallowedHost at /movie/bCZgaGBj
Exception Value: Invalid HTTP_HOST header: 'www.tvmao.com'. You may need to add 'www.tvmao.com' to ALLOWED_HOSTS.

Request information:
USER: [unable to retrieve the current user]

GET: No GET data

POST: No POST data

FILES: No FILES data

COOKIES: No cookie data

META:
HTTP_ACCEPT = '*/*'
HTTP_ACCEPT_LANGUAGE = 'zh-cn'
HTTP_CACHE_CONTROL = 'no-cache'
HTTP_CONNECTION = 'Upgrade'
HTTP_HOST = 'www.tvmao.com'
HTTP_REFERER = '/movie/bCZgaGBj'
HTTP_USER_AGENT = 'Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1'
HTTP_X_FORWARDED_FOR = '27.124.12.23'
HTTP_X_REAL_IP = '27.124.12.23'
PATH_INFO = '/movie/bCZgaGBj'
QUERY_STRING = ''
REMOTE_ADDR = '127.0.0.1'
REMOTE_HOST = '127.0.0.1'
REMOTE_PORT = 44058
REQUEST_METHOD = 'GET'
SCRIPT_NAME = ''
SERVER_NAME = '127.0.0.1'
SERVER_PORT = '8000'
wsgi.multiprocess = True
wsgi.multithread = True

Here's how I've tried to simulate the attack using raw http requests and netcat:

me@linuxmachine:~$ cat raw.http 
GET /dashboard/ HTTP/1.1
Host: serverA.com
Host: test.com
Connection: close

me@linuxmachine:~$ cat raw.http | nc A.B.C.D 80
HTTP/1.1 400 Bad Request
Server: nginx/1.18.0 (Ubuntu)
Date: Fri, 27 Jan 2023 15:05:13 GMT
Content-Type: text/html
Content-Length: 166
Connection: close

<html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

If I send my correct serverA.com as the host header, I get a 301 (redirecting to https). If I send an incorrect host header (e.g. test.com) I get an empty response (expected). If I send two host headers (correct and incorrect) I get a 400 bad request If I send the correct host, but to port 443, I get a 400 plain HTTP sent to HTTPS port...

How do I simulate a request to get past nginx to my upstream serverA like the bots do? And how do I block it with nginx?

Thanks!

Kaustubh Bhalerao
  • 71
  • 5

1 Answers1

1

There is something magical about asking SO. The process of writing makes the answer appear :)

To my first question above, of simulating the spoof, I was able to just use curl in the following way:

me@linuxmachine:~$ curl -H "Host: A.B.C.D" https://example.com

I'm pretty sure I've tried this before but not sure why I didn't try this exact spell (perhaps I was sending a different header, like Http-Host: or something)

With this call, I was able to trigger the error as before, which made it easy to test the nginx configuration and answer the second question.

It was clear that the spoof was coming on 443, which led me to this very informative post on StackExchange

This also explained why we can't just listen 443 and respond with a 444 without first having traded SSL certificates due to the way SSL works.

The three options suggested (happrox, fake cert, and the if($host ...) directive might all work, but the simplest I think is the last one. Since this if( ) is not within the location context, I believe this to be ok.

My new serverA block looks like this:

server {
  server_name serverA.com www.serverA.com;
  client_max_body_size 10M;

  ## This fixes it
  if ( $http_host !~* ^(serverA\.com|www\.serverA\.com)$ ) {
        return 444;
  }
  ## and it's not inside the location context...

  location / {
    proxy_pass  http://upstream;
    proxy_http_version 1.1;
   ...
Kaustubh Bhalerao
  • 71
  • 5