NGINX double reverse proxy

Question

I am experiencing some strange problems with my reverse proxy.

My setup looks like this: setup

Now, hosts inside my internal network can reach the app running on localhost:3000 on server srv1.my.domain.org, so I know the config is working. But as soon as I try and access the app from outside the network (over gateway.my.domain.org) I just see the NGINX test page. I do not understand why, since this config I have done for multiple other apps, and they are working.

NOTE: The server_name is the same as the hostname of the physical server.

Here are the configs:

gateway.my.domain.org

path: /etc/nginx/conf.d/app.my.domain.org.conf

upstream srv {
     server srv1.my.domain.org;
}

server {
     listen 443;
     server_name gateway.my.domain.org;

     ssl on;
     ssl_certificate /.../fullchain.pem;
     ssl_certificate_key  /.../privkey.pem;
     ssl_session_cache  builtin:1000  shared:SSL:10m;
     ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;
     ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
     ssl_prefer_server_ciphers on;

     access_log            /var/log/nginx/gateway.access.log;
     error_log            /var/log/nginx/gateway.error.log;

     client_max_body_size 150M;

     location / {
         proxy_set_header        Host $host;
         proxy_set_header        X-Real-IP $remote_addr;
         proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header        X-Forwarded-Proto $scheme;
         proxy_pass              http://srv;
         proxy_read_timeout      90;
     }
}

srv1.my.domain.org

path: /etc/nginx/sites-enabled/app

server {
    listen 80;
    server_name srv1.my.domain.org;

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_pass http://localhost:3000;
    }
}

iwaseatenbyagrue · Accepted Answer · 2017-03-10T15:47:31.640

My config for a similar setup is a little different:

upstream app_backend {
  server 127.0.0.1:3000 ;
}
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name HOSTNAME;
    location / {
        try_files $uri $uri/ @app;
    }
    location @app {


        #proxy_cache general_cache; 
        proxy_pass http://app_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto http;
        proxy_redirect off;
    }
}

In my case, I have everything on one node, but adapting to your case should be a simple matter of creating similar blocks

You are also overriding the X-Real-IP on srv1.my.domain.org, which I am not sure you want to be doing.

Edited to account for update in comments:

Right - I just realised I missed something (or failed to account for it), which sort of relates to the X-Real-IP thing.

I think you may have an issue in part because on gateway.my.domain.org, you have:

proxy_set_header Host $host;

And on gateway.my.domain.org, $host = gateway.my.domain.org.

So you are sending a request to srv1.my.domain.org (from gateway), that has a Host: header of gateway.my.domain.org. Your vhost on srv1 is configured to respond to a Host header of srv1.my.domain.org - and so your vhost config is not being used. The 404 is most likely coming from the default port 80 server of srv1.

One way of fixing this could be:

proxy_set_header Host $proxy_host;

Which also happens to be the default value of the Host field (per https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header).

This should be simple enough to test: configure logging for srv1.my.domain.org to go to some other destination than the default nginx log files. Then check whether the next request you make shows up in your 'custom' log, or the default one.

The issue with X-Real-IP is that on gateway.my.domain.org, $remote_addr is meaningful, but on srv1.my.domain.org, $remote_addr is the internal address of gateway.my.domain.org.

Even after adapting your config, I am still getting the NGINX test page from srv1.my.domain.org when I want to access it over the internet. Would you care to explain why I shouldn't override X-Real-IP? — ChaChaPoly, Mar 10 '17 at 14:13
Gateway is now sending a header of `proxy_set_header Host srv1.my.domain.org;`, but it is not wokring. Now the connection isn't even reaching srv1 and I am not seeing anything in the gateway's logs. — ChaChaPoly, Mar 10 '17 at 14:44
Checking https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header, default for Host is $proxy_host, which I think would solve your issue. By which I mean, you could remove the `proxy_set_header Host` entirely. I am not sure why using a static string isn't working, but vaguely recall doing things that way could be problematic. — iwaseatenbyagrue, Mar 10 '17 at 15:45

NGINX double reverse proxy

1 Answers1