Setting appropriate HTTP Content-Length header in WSGI middleware after manipulating content

Question

I have a middleware set up in a Werkzeug based app to do some JSON escaping and manipulation for me (especially prefixing JSON with an escape string for an Angular-based REST-client).

I'd like to keep the whole logic in the middleware layer and don't add any tricks to my base view classes or my base app.

Because my middleware manipulates the content I strip the Content-Length header from headers, but I want to be a good netizen and provide the client with that information.

Unfortunately at the point where I have manipulated the content there seems to be no way to adjust headers anymore. Do I have to do this further down the pipeline? Wrap a second middleware around it?

Here's the code to the middleware:

class ContentManipulatingMiddle(object):
    def __init__(self, app):
        self.app = app

    def __call__(self, environ, start_response):
        app = self.app

        def start_unpack_data(status, response_headers, exc_info=None):
            # we need to strip content-length
            response_headers = [ (name, value)
                for name, value in response_headers
                if name.lower() != 'content-length' ]

            return start_response(status, response_headers, exc_info)

        app_iter = app(environ, start_unpack_data)

        data = []
        for item in app_iter:
            # do some content manipulation
            data.append(manipulate_content(item))

        # content length has changed, i should reset the content-length header
        # but at this point, how?

        return data

score 1 · Answer 1 · answered May 26 '20 at 14:26

Got able to change the headers by not allowing the inner app to start_response, providing it a collector dummy function instead:

class MyMiddleware:
    def __init__(self, app):
        self.app = app

    def __call__(self, environ, start_response):
        inner_status = None
        inner_headers = []
        inner_exc_info = None

        def start_response_collector(status, headers, exc_info=None):
            # Just collects the inner response headers, to be modified before sending to client
            nonlocal inner_status, inner_headers, inner_exc_info
            inner_status = status
            inner_headers = headers
            inner_exc_info = exc_info
            # Not calling start_response(), as we will modify the headers first.
            return None

        # populates the inner_* vars, as triggers inner call of the collector closure
        response_iter = self.app(environ, start_response_collector)

        # removes the content-length, if exists
        headers = [(k, v) for k, v in inner_headers if k.lower() != 'content-length']

        inner_body = b"".join(response_iter)

        ### MANIPULATE YOUR `inner_body` HERE ###
        # E.g. producing a final_body
        final_body = b'DEADBEEF'

        final_body_length = len(final_body)
        headers.append(('Content-Length', str(final_body_length)))

        # Remember to send the modified headers!
        start_response(inner_status, headers, inner_exc_info)
        return final_body

Could also be a reusable wrapping middleware by just _not_ manipulating the `inner_body`. Left as an exercise to the reader. — alanjds, May 26 '20 at 14:44

score -1 · Accepted Answer · edited Jan 17 '14 at 10:49

-1

You dont need to worry about deleting/adding/changing content-length header, that will be auto handled by the server when it sends out the response. Sending a wrong length header might create problems for your website viewers / internet browsers.

You can test the Content-length headers here - http://www.dekho-ji.com/website-diagnostics-curl-http-request-response-headers-online

edited Jan 17 '14 at 10:49

Lightness Races in Orbit

378,754
76
643
1,055

answered Jul 04 '13 at 14:41

Tech Consultant

374
1
7

Ok, I wasn't quite aware of that. Werkzeug development server didn't add any content-length headers and didn't correct the wrong content-length. So I definitely should strip a wrong content-length from my response. Still, the problem with manipulating headers after manipulating content remains. It seems weird that WSGI middleware seems to offer no possibility to do so. – room2web Jul 05 '13 at 10:34
see this SO post for adding http headers to WSGI middleware - http://stackoverflow.com/questions/3859097/how-to-add-http-headers-in-wsgi-middleware – Tech Consultant Jul 05 '13 at 12:03
I read that post. Problem is, headers are manipulated in start_unpack_data which call be called further up the middleware stack while content is manipulated after calling app (which calls start_unpack_data). Therefore in start_unpack_data there is no possibility to set headers which depend on the outcome of content manipulation in the same middleware. – room2web Jul 05 '13 at 14:34

Setting appropriate HTTP Content-Length header in WSGI middleware after manipulating content

2 Answers2