How to make a library asynchronous in python

Question

In my job they "leverage" tornado but they have no asynchronous libraries. What makes a library asynchronous so that it can be better suited to something like tornado? Are there any good examples or I guess is there somthing you do in __enter__ or __exit__ that can signal that you are not blocking?

I'm finding it difficult to scratch together some materials.

Answer 1

If your libraries are not asynchronous and do not support being run in the tornado ioloop, then the only thing you can do is run these tasks in other threads.

Essentially, there are two options, depending on whether you want to receive an return value or not:

1. you put your work in a queue and some threads pull work from the queue and work these tasks off ==> no return value
2. or you use the executor library (python 3.2) and send the work into the executor, add a callback to signal that the task is finished and deliver control back to tornado's ioloop (by another callback hung into the loop).

If your_task_func is the sync-task you want offload to another thread essentially do the following:

def callback(future):
    # here you want to do some stuff with the value future.result()

EXECUTOR.submit(
        your_task_func
        ).add_done_callback(
            lambda future: tornado.ioloop.IOLoop.instance().add_callback(
                partial(callback, future)))

More details about this can be found in this nice write-up.

regards markus

Answer 2

Asynchronous--- do you mean threading? If you want to run some code concurrently, you can use the threading module (or the more low-level thread module), which is already built into the standard library. For example:

import threading
import time

def counter():
    c = 0
    while True:
        print c
        time.sleep(1)
        c += 1

counterThread = threading.Thread(target=counter, name="counter")
counterThread.daemon = True   # if False (default), Python interpreter won't quit until the thread ends
counterThread.start()

The lock objects are implemented with __enter__ and __exit__ so that you can use the with keyword, as per your question.

See also third-party threading libraries.