Having problems running a long blocking function in a thread in Tornado

Question

I am very new to Tornado. Was just seeing how to handle a request which blocks in Tornado. I run the blocking code in a separate thread. However the main thread still blocks till the threaded function finishes. I am not using gen.coroutine here, but have tried that and the result is the same

counter = 0

def run_async(func):
    @wraps(func)
    def function_in_a_thread(*args, **kwargs):
        func_t = Thread(target=func, args=args, kwargs=kwargs)
        func_t.start()
    return function_in_a_thread

def long_blocking_function(index, sleep_time, callback):
    print "Entering run counter:%s" % (index,)
    time.sleep(sleep_time)
    print "Exiting run counter:%s" % (index,)

    callback('keyy' + index)


class FooHandler(tornado.web.RequestHandler):

    @web.asynchronous
    def get(self):

        global counter
        counter += 1
        current_counter = str(counter)

        print "ABOUT to spawn thread for counter:%s" % (current_counter,)
        long_blocking_function(
            index=current_counter,
            sleep_time=5, callback=self.done_waiting)


        print "DONE with the long function"

    def done_waiting(self, response):
        self.write("Whatever %s " % (response,))
        self.finish()


class Application(tornado.web.Application):
    def __init__(self):
        handlers = [(r"/foo", FooHandler),
                    ]

        settings = dict(
            debug=True,
        )

        tornado.web.Application.__init__(self, handlers, **settings)


def main():
    application = Application()
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

if __name__ == "__main__":
    main()

When I issue back to back requests the FooHandler blocks and does not recieve any requests till the long_blocking_function finishes. So I end up seeing something like

ABOUT to spawn thread for counter:1
Entering run counter:1
Exiting run counter:1
DONE with the long function
ABOUT to spawn thread for counter:2
Entering run counter:2
Exiting run counter:2
DONE with the long function
ABOUT to spawn thread for counter:3
Entering run counter:3
Exiting run counter:3
DONE with the long function

I was expecting something along these lines(as I am issuing multiple requests before the first call to long_blocking_function finishes) but am only seeing trace similar to above

ABOUT to spawn thread for counter:1
DONE with the long function
ABOUT to spawn thread for counter:2
DONE with the long function
ABOUT to spawn thread for counter:3
DONE with the long function
ABOUT to spawn thread for counter:4
DONE with the long function

I have looked at Tornado blocking asynchronous requests and tried both the solutions. But both of them are blocking when I run them with back to back requests to the same handler. Can somebody figure out what I am doing wrong? I know tornado doesn't do well with multithreading, but I should be able to run a new thread from it, in a non-blocking way.

score 5 · Accepted Answer · answered Oct 15 '13 at 12:48

5

Tornado plays well with the concurrent.futures library (there's a Python 2.x backport available), so you could hand off your long running requests to a thread pool by using a ThreadPoolExecutor.

This technique works pretty well - we use it to handle long-running database operations. Of course, in the real world you'd also want to handle timeouts and other exceptions in a robust and graceful manner, but I hope this example is enough to illustrate the idea.

def long_blocking_function(index, sleep_time, callback):
    print ("Entering run counter:%s" % (index,))
    time.sleep(sleep_time)
    print ("Exiting run counter:%s" % (index,))
    return "Result from %d" % index


class FooHandler(tornado.web.RequestHandler):
    @tornado.gen.coroutine
    def get(self):
        global counter
        counter += 1
        current_counter = str(counter)

        print ("ABOUT to spawn thread for counter:%s" % (current_counter,))
        result = yield self.executor.submit(long_blocking_function,
                                            index=current_counter,
                                            sleep_time=5)
        self.write(result)
        print ("DONE with the long function")

answered Oct 15 '13 at 12:48

R Hyde

10,301
1
32
28

Thanks Rod. I am trying this. Ideally I would want to use the gen.coroutine rather than having get and callback functions. But this blocks. When I issue back to back requests, each request blocks till the previous request's async task finishes. Here is the [BarHandler based on coroutines](https://gist.github.com/simplyvikram/6997323) code. – sinv Oct 15 '13 at 19:41
I've taken the liberty of [forking your gist](https://gist.github.com/badlydrawnrod/7003909). I've tested it using curl and it works as expected. – R Hyde Oct 16 '13 at 07:31
1

It worked!! Thanks Rod. The browser was queing up requests. It worked fine with curl. Got both FooHandler and BarHandler working based on that. – sinv Oct 18 '13 at 16:54
what would be the effect of not using yield on the excecutor call? I have a similar situation where if I use yield, my program crashes, but without it it holds fine... see [here](http://stackoverflow.com/questions/34745117/design-of-asynchronous-request-and-blocking-processing-using-tornado) – jimijazz Jan 12 '16 at 14:47

score 1 · Answer 2 · edited May 23 '17 at 11:59

1

when you offload a task to another thread, you register a callback in the event-loop that is called once the task is finished. The answers are in this thread: How to make a library asynchronous in python.

regards m.

edited May 23 '17 at 11:59

Community

1
1

answered Oct 14 '13 at 14:11

fricke

1,330
1
11
21

Thanks for the response. It was really helpful. Based on that, I am now adding the callback to the ioloop which is called when the non-main thread finishes, but its still blocking new requests. New requests are only processed once previous request's async worker thread finishes. Here is the [link](https://gist.github.com/simplyvikram/6997323) for FooHandler code – sinv Oct 15 '13 at 19:35
I not sure why this happens - at first glance it should do the job. One thought I'm having is that perhaps the threadpool, which is instantiated inside the request causes the problem. Perhaps this blocks the method until the started worker-thread terminates (caused by the pending garbarge collection of executor variable. – fricke Oct 15 '13 at 21:21
It was working fine. It was my browser which was queing up requests. Thanks for your help though. [lbolla's article](http://lbolla.info/blog/2013/01/22/blocking-tornado) is a good read, but thought the recommended way for tornado is now co-routines instead of callbacks – sinv Oct 18 '13 at 17:00

score 0 · Answer 3 · answered Oct 11 '13 at 23:06

You simply forgot to actually use the run_async decorator you have defined.

import tornado.web
import functools
import threading
import time

counter = 0
def run_async(func):
    @functools.wraps(func)
    def function_in_a_thread(*args, **kwargs):
        func_t = threading.Thread(target=func, args=args, kwargs=kwargs)
        func_t.start()
    return function_in_a_thread


@run_async
def long_blocking_function(index, sleep_time, callback):
    print ("Entering run counter:%s" % (index,))
    time.sleep(sleep_time)
    print ("Exiting run counter:%s" % (index,))
    callback('keyy' + index)

class FooHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        global counter
        counter += 1
        current_counter = str(counter)

        print ("ABOUT to spawn thread for counter:%s" % (current_counter,))
        long_blocking_function(
            index=current_counter,
            sleep_time=5, callback=self.done_waiting)
        print ("DONE with the long function")

    def done_waiting(self, response):
        self.write("Whatever %s " % (response,))
        self.finish()


class Application(tornado.web.Application):
    def __init__(self):
        handlers = [(r"/foo", FooHandler),
                    ]

        settings = dict(
            debug=True,
        )

        tornado.web.Application.__init__(self, handlers, **settings)


def main():
    application = Application()
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

if __name__ == "__main__":
    main()

That wasnt the issue. I was using the @run_async decorator, just forgot to copy it over here. Looks like it was working fine all along, but the browser was queing up requests. Using curl now instead as per Rod's recommendation — sinv, Oct 18 '13 at 16:56

Having problems running a long blocking function in a thread in Tornado

3 Answers3

Linked