6

I'm getting a 504 timeout on my Nginx server: 504 Gateway Time-out. The website allows users to download custom data files. The site works fine under most conditions and users can download some files without issue. The problem is that the data files are only generated when requested. So when complex data is requested the Python backend takes more than a minute to generate the file and start the download. Because there is no response within a minute Nginx returns a 504 Gateway Time-out. The files themselves are CSV files and not very large so the downloads are fast after the backend has generated the file, but I need a way to make Nginx wait longer.

I'm looking for a way to increase the Nginx timeout from 1 minute to about 10 minutes.

So far I've tried:

changing the nginx.conf file to include the following lines between the http {} braces

uwsgi_connect_timeout 75s;
proxy_connect_timeout 600;
proxy_send_timeout 600s;
proxy_read_timeout 600s;
fastcgi_send_timeout 600s;
fastcgi_read_timeout 600s;

I've also tried adding a timeout.conf file to /etc/nginx/conf.d/ as suggested by https://asdqwe.net/blog/solutions-504-gateway-timeout-nginx/:

proxy_connect_timeout       600;
proxy_send_timeout          600;
proxy_read_timeout          600;
send_timeout                600;

After making each change I've restarted the server but the timeout still happens after 1 min. this has been further tested by running a curl -o on the server to bypass any service provider issues. The output was as follows.

$curl -o out-put-file.csv "./localhost/my/url"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   183   100   183   0    0       3      0  0:01:01  0:01:00  0:00:01    48

And this error gets written to error.log 

2018/03/26 09:55:15 [error] 10105#10105: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 142.1.2.3, server: localhost, request: "GET /my/url", upstream: "uwsgi://unix:/tmp/my-applacation.sock", host: "142.1.2.3"

When I run my application without Nginx and run the same curl -o the request takes quite a while (up to 8 minutes depending on the requested data) but it does finish properly without errors or a timeout.

Running $ /usr/sbin/nginx -V 2>&1 | grep conf shows the config file is located at --conf-path=/etc/nginx/nginx.conf

Nginx version: nginx/1.12.2 and nginx/1.4.6 (Ubuntu)

OS version: Ubuntu 16.04.1 and Ubuntu 14.04.5 LTS

Edit:

I have tested changing the number of worker_processes in the config file just to make sure that changes to other things in the config file did get picked up and sure enough changing that number changes the number of workers so the file os the correct file and changes to get picked up by Nginx. Just not changes to the timeouts (or maybe the full http section?). I have also tested setting the timeouts to 30s but they still timeout at 1 minute. In addition, tried putting these settings in sites-enabled/my_site but I still have the same results.

Edit2:

As far as I can tell I'm setting the values right according to the Nginx documentation. uwsgi_connect_timeout

ngx_http_proxy_module

Jeff
  • 223
  • 1
  • 2
  • 15
  • 1
    I don't see your nginx configuration anywhere. Please run `nginx -T` and post the complete output in your question. – Michael Hampton Feb 15 '18 at 20:07
  • @ Michael Hampton assume you mean `nginx -t` ? The output of that is `nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful` – Jeff Feb 16 '18 at 13:08
  • @MichaelHampton to be clear, I do get warning about duplicated settings if I have the same settings in `timeout.conf` and `nginx.conf` but so long as I remove the duplicated settings I get no errors. However the timeout is still happening at 60 seconds. – Jeff Feb 16 '18 at 13:25
  • No, I meant `nginx -T`. – Michael Hampton Feb 16 '18 at 16:28
  • @MichaelHampton when I run `nginx -T` the only output I get is `nginx: invalid option: "T"` also `-T` is not in the man page but `-t` is. What is the `-T` suppose to do? – Jeff Feb 16 '18 at 17:53
  • Oh, right, you're running an ancient version of nginx, sorry. Consider updating to a current stable version. – Michael Hampton Feb 16 '18 at 18:02
  • @Michael Hampton I've spun up a newer version of nginx (1.12.1) but I'm still having the same problem. – Jeff Feb 26 '18 at 13:53
  • I can now run `nginx -T`. When I do I get `nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful # configuration file /etc/nginx/nginx.conf:` and then it prints the full nginx.conf file which includes the changes to the timeouts in it. But I still get a timeout at 1 min – Jeff Feb 26 '18 at 14:02
  • @MichaelHampton the full output of nginx -T can be found at https://ideone.com/Ui9tm0 – Jeff Feb 26 '18 at 14:49
  • What was in the error log? – Michael Hampton Feb 26 '18 at 16:14
  • @MichaelHampton when I run `nginx -T` the following line gets added to /var/log/nginx/error.log `2018/02/26 15:39:18 [info] 9258#9258: Using 32768KiB of shared memory for nchan in /etc/nginx/nginx.conf:71` at first, the file was empty, so I had to change the permissions on the file so nginx could wright to it. Also note, no errors get logged here when I make the request that causes the timeout to happen. It is strange to me that it seems to be complaining about line 71 as line 71 is `# Phusion Passenger config` which should just be a comment – Jeff Feb 26 '18 at 19:16
  • @MichaelHampton I've added more information about the message that is logged in error.log I've also added a few other pieces of information to the question. – Jeff Mar 26 '18 at 13:20

1 Answers1

4

I solved it!!

The problem turned out to be related to uwsgi, while I had set

uwsgi_connect_timeout 75s;

what I actually I needed to set was

uwsgi_read_timeout 600s;
Jeff
  • 223
  • 1
  • 2
  • 15