unable to proxy pass to NGINX server running php spf

Question

I have a main NGINX server to run certs for all my internal servers. I am adding an endpoint for a new NGINX server running php. If I connect directly to the new server the new server works correctly. It's able to connect to the database through my docker network. The cert server is able to connect to all other endpoints correctly.

When I use the endpoint on the cert server that proxy passes to the new server I get 502 Bad Gateway on my browser and

2022/08/16 17:45:56 [error] 9#9: *5 connect() failed (111: Connection refused) while connecting to upstream, client: [public ip], server: , request: "GET /timeclock HTTP/1.1", upstream: "http://172.18.0.4:5003/timeclock", host: "aps.devserver.com:5005"
[public ip] - - [16/Aug/2022:17:45:56 +0000] "GET /timeclock HTTP/1.1" 502 559 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36" "-"
[public ip] - - [16/Aug/2022:17:45:56 +0000] "GET /favicon.ico HTTP/1.1" 200 5430 "https://aps.devserver.com:5005/timeclock" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36" "-"

in the logs of the cert server. There are no logs indicating a connection in the php server.

Here is the config for my cert server the /timeclock location is the important bit

upstream client {
  server aps-frontend:80;
}

upstream server {
  server aps-backend:4625;
}

server {
  listen 80;
  return 301 https://$host:5001$request_uri;
}

server {
  listen 443;
  ssl on;
  ssl_certificate /etc/ssl/certs/apscert.pem;
  ssl_certificate_key /etc/ssl/certs/apskey.pem;

  location / {
    proxy_pass http://client;
  }

  location /api {
          return 302 /api/;
  }


  location /api/ {
    proxy_pass http://server/;
  }

  location /timeclock {
    proxy_pass http://timeclock:5003;
  }
}

Here is the config for the php server

worker_processes auto;
error_log stderr warn;
pid /run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    include mime.types;
    default_type application/octet-stream;

    # Define custom log format to include reponse times
    log_format main_timed '$remote_addr - $remote_user [$time_local] "$request" '
                          '$status $body_bytes_sent "$http_referer" '
                          '"$http_user_agent" "$http_x_forwarded_for" '
                          '$request_time $upstream_response_time $pipe $upstream_cache_status';

    access_log /dev/stdout main_timed;
    error_log /dev/stderr notice;

    keepalive_timeout 65;

    # Write temporary files to /tmp so they can be created as a non-privileged user
    client_body_temp_path /tmp/client_temp;
    proxy_temp_path /tmp/proxy_temp_path;
    fastcgi_temp_path /tmp/fastcgi_temp;
    uwsgi_temp_path /tmp/uwsgi_temp;
    scgi_temp_path /tmp/scgi_temp;

    # Default server definition
    server {
        listen [::]:8080 default_server;
        listen 8080 default_server;
        server_name _;

        sendfile off;
        tcp_nodelay on;
        absolute_redirect off;

        root /var/www/html;
        index index.php index.html;

        location / {
            # First attempt to serve request as file, then
            # as directory, then fall back to index.php
            try_files $uri $uri/ /index.php?q=$uri&$args;
        }

        # Redirect server error pages to the static page /50x.html
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
            root /var/lib/nginx/html;
        }

        # Pass the PHP scripts to PHP-FPM listening on php-fpm.sock
        location ~ \.php$ {
            try_files $uri =404;
            fastcgi_split_path_info ^(.+\.php)(/.+)$;
            fastcgi_pass unix:/run/php-fpm.sock;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            fastcgi_param SCRIPT_NAME $fastcgi_script_name;
            fastcgi_index index.php;
            include fastcgi_params;
        }

        location ~* \.(jpg|jpeg|gif|png|css|js|ico|xml)$ {
            expires 5d;
        }

        # Deny access to . files, for security
        location ~ /\. {
            log_not_found off;
            deny all;
        }

        # Allow fpm ping and status from localhost
        location ~ ^/(fpm-status|fpm-ping)$ {
            access_log off;
            allow 127.0.0.1;
            deny all;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            include fastcgi_params;
            fastcgi_pass unix:/run/php-fpm.sock;
        }
    }
    
    gzip on;
    gzip_proxied any;
    gzip_types text/plain application/xml text/css text/js text/xml application/x-javascript text/javascript application/json application/xml+rss;
    gzip_vary on;
    gzip_disable "msie6";
    
    # Include other server configs
    include /etc/nginx/conf.d/*.conf;
}

Answer 1

2022/08/16 17:45:56 [error] 9#9: *5 connect() failed (111: Connection refused) while connecting to upstream, client: [public ip], server: , request: "GET /timeclock HTTP/1.1", upstream: "http://172.18.0.4:5003/timeclock", host: "aps.devserver.com:5005"

This error means that either there is no service running at 172.18.0.4 port 5003 or there is a firewall that blocks connection attempts there.

Please double-check your application server that it is running the software.