How do I configure my uWsgi server to protect against the Unreadable Post Error?

Question

This is the problem:

  File "/app/.heroku/python/lib/python2.7/site-packages/django/utils/six.py", line 535, in next
    return type(self).__next__(self)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/http/multipartparser.py", line 344, in __next__
    output = next(self._producer)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/utils/six.py", line 535, in next
    return type(self).__next__(self)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/http/multipartparser.py", line 406, in __next__
    data = self.flo.read(self.chunk_size)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/http/request.py", line 267, in read
    six.reraise(UnreadablePostError, UnreadablePostError(*e.args), sys.exc_info()[2])
  File "/app/.heroku/python/lib/python2.7/site-packages/django/http/request.py", line 265, in read
    return self._stream.read(*args, **kwargs)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 59, in read
    result = self.buffer + self._read_limited(size - len(self.buffer))
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 47, in _read_limited
    result = self.stream.read(size)
UnreadablePostError: error during read(65536) on wsgi.input

My current configuration reads like this:

[uwsgi]
http-socket = :$(PORT)
master = true
processes = 4
die-on-term = true
module = app.wsgi:application
memory-report = true
chunked-input-limit = 25000000
chunked-input-timeout = 300
socket-timeout = 300

Python: 2.7.x | uWsgi: 2.0.10

And to make the problem even more specific, this is happening when I process images synchronously along with an image upload. I know that ideally I must do this using Celery, but because of a business requirement I am not able to do that. So need to configure the timeout in such a way that it allows me to accept a large image file, process it and then return response.

Any kind of light on the question will be extremely helpful. Thank you.

Did you try to set `buffer-size=65536` in the configuration file — Nguyen Sy Thanh Son, Jan 14 '16 at 07:25
Does this happen for *every* request? Can you provide a minimal example that reproduces the issue? — Phillip, Jan 14 '16 at 08:42
@NguyenSyThanhSon No, I haven't tried to set the buffer size. — Sai Krishna, Jan 14 '16 at 08:50
@Phillip This happens for image sizes greater than 7mb, sometimes even images as less as 5 mb. — Sai Krishna, Jan 14 '16 at 08:50
Can you rule out the possibility that the error is due to a second request? Did you try reading the entire request body into a variable (or temporary file) first and start processing afterwards? — Phillip, Jan 14 '16 at 08:56
@Phillip, cannot rule it out. A log recently from the server: [uwsgi-body-read] Error reading 65536 bytes. Content-Length: 992197 consumed: 131072 left: 861125 message: Client closed connection uwsgi_response_write_body_do() TIMEOUT !!! — Sai Krishna, Jan 15 '16 at 07:57
Well, in that case: Isn't it possible that what you see are simply requests users actively aborted before the upload was completed? — Phillip, Jan 15 '16 at 12:37
You can't recive a message if determine a bytesize( `0xFFFF equal to 65535` so max). Multipart data not a message ! Every message got `EOF`. You can't accept message if reached to maximum byte size. Sentence and text are different(sentence=post_message,text=multipart_data). — dsgdfg, Jan 19 '16 at 12:43

score 3 · Answer 1 · answered Jan 21 '16 at 04:32

The error quoted in the description isn't the full picture; the relevant part is this lot entry:

[uwsgi-body-read] Error reading 65536 bytes … message: Client closed connection uwsgi_response_write_body_do() TIMEOUT

This specific error is being raised because (most probably) the client, or something between it and uWSGI, aborted the request.

There are a number of possible causes for this:

A buggy client
Network-level filtering (DPI or some misconfigured firewall)
Bugs / misconfiguration in the server in front of uWSGI

The last one is covered in the uWSGI docs:

If you plan to put uWSGI behind a proxy/router be sure it supports chunked input requests (or generally raw HTTP requests).

To verify your issue really isn't in uWSGI, try to upload the file via the console on the server hosting your uWSGI application. Hit the HTTP endpoint directly, bypassing nginx/haproxy and friends.

How do I configure my uWsgi server to protect against the Unreadable Post Error?

1 Answers1