nginx Returning 500 for POST Image Request with Django Rest Framework

Question

Preface

I have done a lot of research for solutions to similar problems I have been having, but have still yet to solve this major problem I am experiencing. As a last resort, as a lot of cases seem application-specific, I have decided to create a post here with my configuration in the hope that someone may see a possible solution.

Framework

Django 2.1.7
nginx/1.14.2
djangorestframework==3.9.2

The Problem

On my local setup of my server with the above framework, I can POST an image through my client app (specifically a react-native app, which is likely not important information) just fine. However, when trying to make the same POST request with the same image and URL path to my remote server, it returns the following 500 error (snapshotted from Google Chrome Console debugger).

I have strongly deduced that nginx is the culprit for just image uploads, as the django logs of my remote server show no indication that it has even received the POST request, as my local server does. A normal POST request without image uploading, however, works just fine on both the remote and local server.

What I Have Tried

As found in other posts, I have increased the client_max_body_size of the nginx.conf to a very large number of 2000M. I have also set the following django settings:

FILE_UPLOAD_PERMISSIONS = 0o777

FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440000

DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440000

None of these proposed solutions have worked.

My Config

nginx.conf

user                        www;
worker_processes            auto;

events { worker_connections 1024; }
daemon off;

http {
    server_tokens off;
    sendfile      on;
    include       mime.types;

    default_type  application/octet-stream;

    gzip              on;
    gzip_http_version 1.0;
    gzip_proxied      any;
    gzip_min_length   500;
    gzip_disable      "MSIE [1-6]\.";
    gzip_types        text/plain text/xml text/css
                    text/comma-separated-values
                    text/javascript
                    application/x-javascript
                    application/atom+xml;

    # List of application servers
    upstream app_server {
        server 127.0.0.1:9090;
    }

    # PRODUCTION (also default)
    server {
        # Using an alias, not the actual server name
        server_name my-remote-server.com;

        # Running port
        listen 8080 default_server;
        client_max_body_size 100M;
        keepalive_timeout    15;

        # Principle difference between production
        if ($http_x_forwarded_proto = "http") {
            return 301 https://$host$request_uri;
        }

        # Don't serve .py files
        location ~ (\.py$|\.pyc$) {
            return 403;
        }

        # Static assets served directly
        location /static/ {
            alias           /var/app/myserver/src/site/static/;
            access_log      off;
            log_not_found   off;
        }

        location /media/ {
            alias           /var/app/myserver/src/myserver/media/;
            access_log      off;
            log_not_found   off;
        }

        # Proxying the connections
        location / {
            proxy_pass              http://app_server;
            proxy_redirect          off;
            proxy_set_header        Host $host;
            proxy_set_header        X-Real-IP $remote_addr;
            proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header        X-Forwarded-Host $server_name;
        }
    }
}

The Big Questions

1. Why might nginx return a 500 error?
2. Is there a way to get a more verbose error message from nginx to know why it is a returning a 500 error?
3. Is my config/settings set-up in a way that I should expect uploading of images, or could there be improvements or missing pieces that may fix this problem?

Any thoughts or feedback would be extremely appreciated. Thank you.

Answer 1

After much, much research and attempts through trial/error, I have found a solution that works for my situation.

In nginx.conf in the http object, set the following properties:

// nginx.conf

...
http { 
    ...
    client_max_body_size 50M; // this can be whatever max size you want
    client_body_temp_path /tmp/client_body_temp;
    ...
}
...

According to the nginx docs, client_body_temp_path defines a directory for storing temporary files holding client request bodies (ref: http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_temp_path). Once a request becomes large (i.e. through an image upload), the nginx server requires a place to buffer as it processes the entire request. Apparently my server did not have permissions to the default temp path client_body_temp, so I manually created a folder with the path /tmp/client_body_temp with permissions chmod 744, and the server finally responded with HTTP 200. (/tmp has the added benefit of automatically cleaning itself up after a certain amount of time).

Note: After changing the nginx.conf file, you will need to run nginx -s reload to refresh the server with the new configuration (use nginx -t beforehand to check it is well-formatted).

Definitely hope this helps people as much as it was a huge relief for me!