How to use async tornado API inside tornado.wsgi.WSGIContainer?

Question

I try to use custom WSGIContainer that should work with asynchronous operations:

from tornado import httpserver, httpclient, ioloop, wsgi, gen

@gen.coroutine
def try_to_download():
    response = yield httpclient.AsyncHTTPClient().fetch("http://www.stackoverflow.com/")
    raise gen.Return(response.body)


def simple_app(environ, start_response):
    res = try_to_download()

    print 'done: ', res.done()
    print 'exec_info: ', res.exc_info()

    status = "200 OK"
    response_headers = [("Content-type", "text/html")]
    start_response(status, response_headers)
    return ['hello world']


container = wsgi.WSGIContainer(simple_app)
http_server = httpserver.HTTPServer(container)
http_server.listen(8888)
ioloop.IOLoop.instance().start()

But it isn't work. It seems that application does not wait try_to_download function result. Also code below doesn't work:

from tornado import httpserver, httpclient, ioloop, wsgi, gen


@gen.coroutine
def try_to_download():
    yield gen.Task(httpclient.AsyncHTTPClient().fetch, "http://www.stackoverflow.com/")


def simple_app(environ, start_response):

    res = try_to_download()
    print 'done: ', res.done()
    print 'exec_info: ', res.exc_info()

    status = "200 OK"
    response_headers = [("Content-type", "text/html")]
    start_response(status, response_headers)
    return ['hello world']


container = wsgi.WSGIContainer(simple_app)
http_server = httpserver.HTTPServer(container)
http_server.listen(8888)
ioloop.IOLoop.instance().start()

Do you have any ideas why it isn't works? Python version that I use is 2.7.

P.S. You may ask me why I don't want to use native tornado.web.RequestHandler. The main reason is that I have custom python library (WsgiDAV) that produce WSGI interface and allows to write custom adapters which I what to make asynchronous.

score 4 · Answer 1 · answered Jan 30 '14 at 15:23

4

WSGI doesn't work with async.

In general, for a function to wait for a Tornado coroutine to finish, the function itself must be a coroutine and must yield the coroutine's result:

@gen.coroutine
def caller():
    res = yield try_to_download()

But of course a WSGI function like simple_app cannot be a coroutine because WSGI doesn't understand coroutines. A more thorough explanation of the incompatibility between WSGI and async is in the Bottle documentation.

If you must support WSGI, don't use Tornado's AsyncHTTPClient, use a synchronous client like the standard urllib2 or PyCurl instead. If you must use Tornado's AsyncHTTPClient, don't use WSGI.

answered Jan 30 '14 at 15:23

A. Jesse Jiryu Davis

23,641
4
57
70

Thanks for the answer. It seems you are right. But presently I don't know why such feature as WSGIContainer may be used in tornado application if it can't works with tornado asynchronous programming API. So it blocks the main application. – Dmitry Jan 30 '14 at 18:12
3

You can use WSGIContainer as long as the WSGI app you're using is fast enough that you don't mind blocking the event loop. It's especially useful when there is some benefit to having the WSGI app in the same process as a Tornado app -- I use WSGIContainer to plug Dowser into my Tornado apps to detect memory leaks, for example. – Ben Darnell Jan 31 '14 at 03:48

How to use async tornado API inside tornado.wsgi.WSGIContainer?

1 Answers1