0

I got a Nginx over Nginx configuration on my server.

Server scheme

Nginx A -> Nginx B -> PHP FPM

(If you want to know the reason hidden below, it's because one of the two is in a Docker container.)

Problem description

When a client wants to download a big file, Nginx A receive the request, forward it to Nginx B which send it to PHP FPM.

If I send a SIGQUIT signal on Nginx B, it waits until all the requested file has been transmitted, then it closes gracefully. During this time, Nginx A successfully retransmits all the packets to the client.

BUT: when Nginx B has sent its last packet and it closes, Nginx A captures the closed connection of Nginx B and closes the connection with the client before transmitting the last packet to it.

Experiments

Information retrieved during my tests:

  • On average, less than 8 MB is missing from the file sent to the client (tried with small files, and big files like 1 GB or 4 GB)
  • Nginx A log an error upstream prematurely closed connection while sending to client (it shouldn't, it should finish sending the packets to the client and then close this connection without any problem).
  • If I plug the client connection directly into Nginx B, and send a SIGQUIT, the client does receive all its packets before the connection is closed

Configuration

Nginx A configuration file:

  server_name mywebsite.io;

  access_log /var/log/nginx/mywebsite.io.access.log;
  error_log /var/log/nginx/mywebsite.io.error.log error;

  include /etc/nginx/conf.d/server/default.conf;
  include /etc/nginx/conf.d/server/ssl.conf;
  include /etc/nginx/conf.d/server/csp.conf;
  
  location / {
    client_max_body_size 10G;
    client_body_timeout 43200s; # 12h
    proxy_read_timeout 1800s; # 30m
    proxy_max_temp_file_size 0;
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Host $server_name;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_http_version      1.1;
    proxy_set_header        Upgrade $http_upgrade;
    proxy_set_header        Connection "upgrade";
    proxy_pass              http://127.0.0.1:8000;
  }

Nginx B configuration file:

  listen 80;

  root /var/www/html/public;

  server_name _;

  index index.php;

  client_max_body_size 10G;

  location / {
    try_files $uri $uri/ /index.php$is_args$args;
  }

  location ~ \.php$ {
    try_files $uri =404;
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_pass php:9000;
    fastcgi_index index.php;
    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param PATH_INFO $fastcgi_path_info;
    fastcgi_read_timeout 600s; # 10m
  }

I tried to tweak many parameters:

client_max_body_size, client_body_timeout, proxy_read_timeout 1800s, proxy_max_temp_file_size, proxy_buffering, proxy_connect_timeout, proxy_send_timeout, proxy_intercept_errors, proxy_buffers, proxy_busy_buffers_size, proxy_buffer_size, proxy_set_header with empty "Connection"...

3 days lost with that, Thanks if you have an idea to help me ! I want to understand what's going on :/

Doubidou
  • 131
  • 5

0 Answers0