Nginx silently dropping header lines that exceed 1128 bytes

Question

I have Nginx set up as a reverse proxy to a uWSGI application (a Django app). Due to the nature of the content that the Django app is hosting, sometimes the URLs get long. Really long. Because of this, the Referer HTTP header can also sometimes get long.

Here's the problem. Nginx drops the Referer received from the client if it exceeds 1128 bytes (discovered by trial and error). Anything 1128 bytes or smaller gets passed to the upstream server. Everything larger gets dropped. This is a problem because Django's CSRF protection mechanism requires an intact Referer.

Here's the relevant section of my Nginx config. file:

location / {
    proxy_pass http://127.0.0.1:8000;
    proxy_set_header Host              $host;
    proxy_set_header Referer           $http_referer;
    proxy_set_header X-Real-IP         $remote_addr;
    proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

I looked through the Nginx documentation and couldn't find a setting that controlled the maximum length for client headers. Frustrated, I even began pouring over the source code for Nginx, which didn't turn anything useful up either.

What do I need to do to get Nginx to pass Referer unmodified to the upstream server?

Version Information

Ubuntu 14.04 Server 64-bit
nginx/1.4.6 (Ubuntu)
uWSGI 1.9.17.1-debian
Django 1.7.0

Sounds like you've run into a bug. What versions of everything are you running? — Michael Hampton, Oct 22 '14 at 18:18

score 4 · Accepted Answer · edited Apr 13 '17 at 12:14

4

Xavier's answer and the ensuing discussion led me to the real cause of this issue: uWSGI. It appears that uWSGI was stripping the header:

[WARNING] unable to add HTTP_REFERER

The solution was painfully simple - when launching uWSGI, I needed to simply include the following option:

--buffer-size 8192

All of the headers make it through to Django now and CSRF verification succeeds.

edited Apr 13 '17 at 12:14

Community

1

answered Oct 22 '14 at 20:32

Nathan Osman

2,725
7
32
46

score 1 · Answer 2 · answered Oct 22 '14 at 18:19

1

Nginx header read size is controlled by two directives :

client_header_buffer_size [buffer_size] which is the default buffer that would fit in most cases
large_client_header_buffers [count] [buffer_size] which are additionnal buffers allocated on demand if it doesn't fit in client_header_buffer_size buffer. You must tailor this last directive to your case and make sure than buffer_size is greater than any line passed in the request (headers + URL with HTTP method and HTTP version).

answered Oct 22 '14 at 18:19

Xavier Lucas

13,095
2
44
50

I've set `client_header_buffer_size 16386;` and `large_client_header_buffers 4 16386;` but neither seems to make a difference even though the total length of the request is less than 4096 bytes. – Nathan Osman Oct 22 '14 at 18:35
@NathanOsman something in the logs in debug mode ? Do you get a 400/431 status code ? – Xavier Lucas Oct 22 '14 at 18:50
I get a 403 Forbidden status code - which I believe comes from the upstream server. The 403 is triggered by Django because it can't verify the contents of the `Referer` header. – Nathan Osman Oct 22 '14 at 18:55
@NathanOsman Did a quick test with a dummy proxy, nothing of this kind happened. Could you post the full configuration, your curl request and response to nginx, the intermediate curl request and response to your upstream target and your logs in debug mode ? – Xavier Lucas Oct 22 '14 at 19:04
Sure, give me a few minutes to get all of the information together. – Nathan Osman Oct 22 '14 at 19:06
Here's a cURL request that triggers the error: https://dpaste.de/5FrB. Interestingly, if I remove the query string from the URL, I no longer get the same error. (Just a failed CSRF check instead of a complaint about the `Referer` being missing.) Perhaps I was wrong. The error may be related to the length of the query string in the URL. I'll continue to dig deeper. – Nathan Osman Oct 22 '14 at 19:17
@NathanOsman Does the response headers contain something from the upstream ? I hardly doubt nginx is in cause here. – Xavier Lucas Oct 22 '14 at 19:25
1

Ah, good catch. Running uWSGI with verbose logging reveals that it is indeed the culprit: "[WARNING] unable to add HTTP_REFERER". – Nathan Osman Oct 22 '14 at 19:49

Nginx silently dropping header lines that exceed 1128 bytes

Version Information

2 Answers2