1

I'm using nginx as reverse-proxy to be able to access my Django API and also serve static files. My Django API is using gunicorn.

I have an endpoint allowing a user to download a csv file. I followed the instructions here, Streaming large CSV files: https://docs.djangoproject.com/en/2.2/howto/outputting-csv/

Here is the nginx configuration:

upstream docker-api {
    server api;
}

server {
    listen 443 ssl;
    server_name xxxx.com;

    ssl_certificate /path/to/fullchain.pem;
    ssl_certificate_key /path/to//privkey.pem;
    include /path/to/options-ssl-nginx.conf;
    ssl_dhparam /path/to/ssl-dhparams.pem;


    location /static {
        autoindex on;
        alias /static/;
    }

    location /uploads {
        autoindex on;
        alias /uploads/;
    }

    location / {
        proxy_pass         http://docker-api;
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Host $server_name;
        proxy_set_header   X-Forwarded-Proto $scheme;
    }
}

This is the command I use to launch my gunicorn server:

gunicorn my_api.wsgi -b 0.0.0.0:80 --enable-stdio-inheritance -w 2 -t 180 -k gevent

And when I try to download the file, Gunicorn always timeout my request after 3 minutes. It should not timeout a streaming http response.

Jérémy Octeau
  • 689
  • 1
  • 10
  • 26

2 Answers2

3

this error is typically raised by two things:

  1. Nginx timeout: in which you should add these two lines of code to the Nginx file:

    • first, go to the file you created in the following directory (in my case named myproject)

$ nano /etc/nginx/sites-available/myproject

  • the location / part of your file should look like this:(notice the timeout parts):

location / {
        include proxy_params;
        proxy_pass http://unix:/run/gunicorn.sock;
        proxy_connect_timeout 300s;
        proxy_read_timeout 300s;
    }
  • then restart Nginx:

$ systemctl start nginx

  1. gunicorn timeout: go to the exact directory:

$ nano /etc/systemd/system/gunicorn.service

  • add the --timeout 300 to the file, under [service] section. for more help you see the whole file bellow from link:

[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target

[Service]
User=sammy
Group=www-data
WorkingDirectory=/home/sammy/myprojectdir
ExecStart=/home/sammy/myprojectdir/myprojectenv/bin/gunicorn \
          --access-logfile - \
          --workers 3 \
          --timeout 300 \
          --bind unix:/run/gunicorn.sock \
          myproject.wsgi:application

[Install]
WantedBy=multi-user.target
  • next, restart gunicorn:

$ systemctl daemon-reload

$ systemctl restart gunicorn.socket gunicorn.service

Community
  • 1
  • 1
Alireza Gilaki
  • 31
  • 2
2

The problem is your gunicorn command.

Let's take a look:

gunicorn my_api.wsgi -b 0.0.0.0:80 --enable-stdio-inheritance -w 2 -t 180 -k gevent

-t on your gunicorn command stands for timeout, it's in seconds, and your gunicorn worker timeouts because you've set the timeout to 180 seconds (3 minutes).

And when I try to download the file, Gunicorn always timeout my request after 3 minutes.

To solve this issue, you can simply increase the timeout, for example, the following gunicorn command will set the timeout to 5 minutes:

gunicorn my_api.wsgi -b 0.0.0.0:80 --enable-stdio-inheritance -w 2 -t 300 -k gevent

Check the documentation for more information: http://docs.gunicorn.org/en/stable/settings.html#timeout

vremes
  • 654
  • 2
  • 7
  • 10
  • 1
    But what if the file take more than 5 minutes? Is there a way to avoid the timeout during a streaming http connection? I understand that Gunicorn timeout every other requests after 3 minutes but it should not timeout a streaming request – Jérémy Octeau Jun 01 '19 at 21:19
  • Then you have to set up a separate gunicorn instance running the same application or a separate application serving files with a different timeout. As a side note if you are serving files from a hard drive then there may be no reason to use Django, maybe you can serve with nginx directly? – boreq Jun 02 '19 at 00:02
  • There is no way to say that if the request is a Streaming Http request then the timeout doesn't apply to it? – Jérémy Octeau Jun 02 '19 at 00:04
  • As far as i know, there is no way to do that. I also had issue with Gunicorn timeouts, because i had Flask route that took too long to return, i ended up using Redis Queue ([python-rq](https://python-rq.org/)) instead. – vremes Jun 03 '19 at 04:46