nginx server that only proxies to my game servers

Question

I have clients connecting to my WebSockets game server for an online browser game. Game servers are created and destroyed when players start games, and so each server can have a different IP address that I can’t control.

I need the WebSockets to be secure (WSS) so I have an nginx proxy with an SSL certificate. The client receives the IP of the game server but instead of connecting directly (insecure) it goes via my nginx server with the game server IP as a query param.

Here’s the problem: anyone can now use my nginx server to proxy to any IP they choose. What I need is a way to ensure nginx is only proxying to my game servers.

I don’t have control of the game server IPs since it’s an external host, but I own the game server code. My nginx proxy is hosted by me but the game servers are hosted by a provider.

My plan was to have a shared secret key on the game server and nginx and encrypt all traffic with that, but I’m struggling to find out how this could be done.

Here's what I've got done so far (based on this gist):

I created my own self-signed CA certificate:

openssl genrsa -des3 -out rootCA.key 4096
openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.crt

I created a certificate for the game server:

openssl genrsa -out gameserver.key 2048
openssl req -new -key gameserver.key -out gameserver.csr
openssl x509 -req -in gameserver.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out gameserver.crt -days 500 -sha256

I did the same for the nginx server (~~is this needed?~~ note: not needed):

openssl genrsa -out nginx.key 2048
openssl req -new -key nginx.key -out nginx.csr
openssl x509 -req -in nginx.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out nginx.crt -days 500 -sha256

My nginx config is then something like this:

http {
    map $http_upgrade $connection_upgrade {
        default upgrade;
        '' close;
    }

    server {
        listen 443 ssl;
        # these are for game client
        ssl_certificate /etc/letsencrypt/live/mywebsite.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/mywebsite.com/privkey.pem;

        location / {
            if ( $arg_host != "" ) {
                proxy_pass https://$arg_host:$arg_port;
            }
            proxy_ssl_certificate     nginx.crt;
            proxy_ssl_certificate_key nginx.key;

            proxy_ssl_trusted_certificate rootCA.crt;

            proxy_ssl_verify on;
            proxy_ssl_verify_depth 2;

            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $connection_upgrade;
            proxy_set_header Host $host;
            proxy_read_timeout 86400;
        }
    }
}

My game server is a WebSockets server using gameserver.key and gameserver.csr.

However when I try it the nginx error logs show:

upstream SSL certificate verify error: (18:self signed certificate) while SSL handshaking to upstream

I'm not sure if this can work, and where I went wrong? The only article I found mentioning this error suggests the game server certificate isn't trusted but I can't figure out why.

I'm also not sure what value I should be putting for the Common Name when creating certificates (since each game server it on its own IP) and if this is a problem or not.

Is the traffic from nginx to the backend also encrypted? Have you looked at client certificates for the proxy https://docs.nginx.com/nginx/admin-guide/security-controls/securing-http-traffic-upstream/ — Jacob Evans, Dec 01 '20 at 00:27
@MichaelHampton Thanks for the suggestion but it looks like it's based around URLs? I'm not clear how it could work for WebSockets — edbentley, Dec 01 '20 at 14:42
@JacobEvans I tried using client certificates and have updated the question with how far I got. — edbentley, Dec 01 '20 at 15:09
`proxy_ssl_verify_depth 2;` < make this 1 as you only have a root ca (not a root ca and an intermediate ca). Also did you sign the certificate with the IP in the SANs? https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/sssd-ldap-domain-ip — Jacob Evans, Dec 01 '20 at 15:13
if the IP doesn't match the SSL cert (or you aren't creating the cert on demand for each IP/Host) you want ssl verify off, and just use the mechanism for client cert auth and not backend cert verification, follow? — Jacob Evans, Dec 01 '20 at 15:18
@MichaelHampton Right of course :D. Looking it up I haven't quite understood how it would work in my setup, I'll keep investigating. — edbentley, Dec 01 '20 at 15:19
@JacobEvans Ah I wasn't creating the certificate on demand. When you say "doesn't match", do you mean the IP has to be the field entered for CN? The problem is if I disable ssl verification, then anyone else can piggyback onto my proxy since the host is a query param. — edbentley, Dec 01 '20 at 15:22
override the name field with the CN value http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_name — Jacob Evans, Dec 01 '20 at 15:29
@JacobEvans Thanks for your continued help, I think I tried that before, bit I did it again just now and still the same error. :( To be honest, if CN is the problem I'd expect a different error message? — edbentley, Dec 01 '20 at 15:35
Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/116855/discussion-between-jacob-evans-and-edbentley). — Jacob Evans, Dec 01 '20 at 15:46

Jacob Evans · Accepted Answer · 2020-12-01T17:23:50.927

What I would suggest is setting up an internal certificate authority for the backends and require that your backends can be verified with that certificate authority, you can either bootstrap these certs to include their IP or override the IP with a common hostname (which I think would suffice).

create an internal only CA, call it "Gameserver Backend"
create an internal only server certificate, call it "gameserver.auth.backend"
use the gameserver.auth.backend as your cert on your websocket node program
tell nginx to validate against that, overriding the common name with the one you specified (instead of the IP)

http {
    map $http_upgrade $connection_upgrade {
        default upgrade;
        '' close;
    }

    server {
        listen 443 ssl;
        # these are for game client
        ssl_certificate /etc/letsencrypt/live/mywebsite.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/mywebsite.com/privkey.pem;

        location / {
            if ( $arg_host != "" ) {
                proxy_pass https://$arg_host:$arg_port;
            }

            proxy_ssl_verify on;
            proxy_ssl_verify_depth 2;
            proxy_ssl_name gameserver.auth.backend;
            proxy_ssl_server_name on;
            proxy_ssl_trusted_certificate GameserverCA.crt;

            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $connection_upgrade;
            proxy_set_header Host $host;
            proxy_read_timeout 86400;
        }
    }
}

Ah it was a problem with the CN, which I didn’t pay enough attention to before. It works now, thank you! — edbentley, Dec 02 '20 at 00:39

nginx server that only proxies to my game servers

1 Answers1