"I/O operation on closed file" with StringIO in Django view test

Question

I inherited the following Django view code, used by another webservice to serve downloadable versions of output data:

def index(request):
    # ... (snip) ...
    data = base64.decodestring(request.POST['data'])
    filename = request.POST['filename']

    wrapper = FileWrapper(StringIO(data))

    response = HttpResponse(wrapper, content_type=guess_type(str(filename))[0])

    response['Content-Length'] = len(data)
    response['Content-Disposition'] = "attachment; filename=" + filename

    return response

The function itself — written against Django 1.0 — still works fine after upgrading to 1.5. Unfortunately, the test that covers this view is now failing:

    def testDownload(self):
        self.client.login(username='test', password='test')

        real = 'abc' * 100
        data = base64.encodestring(real)
        response = self.client.post("/api/v1/download/", {'data': data, 'filename': 'out.jpg'})

        self.assertEqual(real, response.content)
        self.assertEqual(response['Content-Disposition'], 'attachment; filename=out.jpg')

and the error:

Traceback (most recent call last):
  File "/home/fred/.secret_projects/final/gerbils/tests/amf.py", line 548, in testDownload
    self.assertEqual(real, response.content)
  File "/home/fred/.virtualenvs/cunning_plot/lib/python2.7/site-packages/django/http/response.py", line 282, in content
    self._consume_content()
  File "/home/carl/.virtualenvs/cunning_plot/lib/python2.7/site-packages/django/http/response.py", line 278, in _consume_content
    self.content = b''.join(self.make_bytes(e) for e in self._container)
  File "/home/carl/.virtualenvs/cunning_plot/lib/python2.7/site-packages/django/http/response.py", line 278, in <genexpr>
    self.content = b''.join(self.make_bytes(e) for e in self._container)
  File "/usr/lib64/python2.7/wsgiref/util.py", line 30, in next
    data = self.filelike.read(self.blksize)
  File "/usr/lib64/python2.7/StringIO.py", line 127, in read
    _complain_ifclosed(self.closed)
  File "/usr/lib64/python2.7/StringIO.py", line 40, in _complain_ifclosed
    raise ValueError, "I/O operation on closed file"
ValueError: I/O operation on closed file

So.. any ideas? I can't see anything in testDownload() or index() which would necessarily "close" the StringIO before it needs to be read. And if there were something, wouldn't it affect the non-test situation as well?

Very confused. Help appreciated.

Answer 1

An easy way to see where close is being called is to just subclass StringIO and put a breakpoint in the close function.

class CustomStringIO(StringIO):
    def close(self):
        import pdb; pdb.set_trace()
        super(CustomStringIO, self).close()

The stack for this is

-> response = self.client.post("/test/", {'data': data, 'filename': 'out.jpg'})
  ...\venv\lib\site-packages\django\test\client.py(463)post()
-> response = super(Client, self).post(path, data=data, content_type=content_type, **extra)
  ...\venv\lib\site-packages\django\test\client.py(297)post()
-> return self.request(**r)
  ...\venv\lib\site-packages\django\test\client.py(406)request()
-> response = self.handler(environ)
  ...\venv\lib\site-packages\django\test\client.py(119)__call__()
-> response.close()                    # will fire request_finished
  ...\venv\lib\site-packages\django\http\response.py(233)close()
-> closable.close()
> \testapp\views.py(11)close()
-> super(CustomStringIO, self).close()

It looks like the test client is closing the response, which in turn calls close on FileWrapper which then calls close on StringIO. This is all before you actually get to response.content.

Is there a reason why you need FileWrapper? As HttpResponse takes in string content and base64.decodestring returns a binary string, it seems you could just pass data directly to HttpResponse instead of having to create a StringIO and FileWrapper.