Two independent async loops in Python

Question

What would be a good approach to execute two asynchronous loops running in parallel in Python, using async/await?

I've thought about something like the code below, but can't wrap my head around how to use async/await/EventLoop in this particular case.

import asyncio

my_list = []

def notify():
    length = len(my_list)
    print("List has changed!", length)

async def append_task():
    while True:
        time.sleep(1)
        await my_list.append(random.random())
        notify()

async def pop_task():
    while True:
        time.sleep(1.8)
        await my_list.pop()
        notify()

loop = asyncio.get_event_loop()
loop.create_task(append_task())
loop.create_task(pop_task())
loop.run_forever()

Expected output:

$ python prog.py
List has changed! 1 # after 1sec
List has changed! 0 # after 1.8sec
List has changed! 1 # after 2sec
List has changed! 2 # after 3sec
List has changed! 1 # after 3.6sec
List has changed! 2 # after 4sec
List has changed! 3 # after 5sec
List has changed! 2 # after 5.4sec

So what output are you getting instead? `BaseEventLoop` is not a concrete loop implementation; use `asyncio.get_event_loop()` to get a concrete loop for your platform. — Martijn Pieters, Dec 30 '16 at 15:28
Also, `list` operations are not awaitable. `time.sleep()` won't yield to other coroutines. — Martijn Pieters, Dec 30 '16 at 15:29

score 11 · Accepted Answer · answered Dec 30 '16 at 15:32

11

this works fine:

note: you wanted to await fast non-io bound operations (list.append and list.pop that are not even coroutines); what you can do is awaitasyncio.sleep(...) (which is a coroutine and yield control back to the caller):

import asyncio
import random

my_list = []


def notify():
    length = len(my_list)
    print("List has changed!", length)

async def append_task():
    while True:
        await asyncio.sleep(1)
        my_list.append(random.random())
        notify()

async def pop_task():
    while True:
        await asyncio.sleep(1.8)
        my_list.pop()
        notify()


loop = asyncio.get_event_loop()
cors = asyncio.wait([append_task(), pop_task()])
loop.run_until_complete(cors)

time.sleep itself is blocking and does not play nicely with await.

answered Dec 30 '16 at 15:32

hiro protagonist

44,693
14
86
111

could you explain what to do if I do have a blocking function (such as time.sleep())? All examples use this asyncio.sleep(), but I have actual long computations (or i/o bound stuff), and it's unclear to me how to wrap it such that this example would work. Thanks a lot. – Fab May 13 '17 at 03:12
@Fab: asyncio will not help with cpu-bound functions. and if your i/o-bound stuff is not built on asyncio it will not work either (that is why `time.sleep()` does not work, but `asyncio.sleep()` does.) you need to call coroutines that yield the program flow back to the caller to get a benefit from the async loop. – hiro protagonist May 13 '17 at 08:20
Thanks a lot, hiro. That clears things up considerably. I've been looking in the wrong place then! – Fab May 13 '17 at 21:29
Asyncio not available on 2.7. – rbonallo Aug 21 '18 at 11:01
@rbonallo that is true. but as the question is about `asyncio` it is about python 3. or what is your point? – hiro protagonist Aug 21 '18 at 11:15
No point, just to save people spending 5 mins pip installing, googling error and realizing it is sadly not available on 2.7. – rbonallo Aug 21 '18 at 11:17
@rbonallo ah, ok! thanks for clarifying! will edit the question and add the python 3 tag. that may make it a bit more explicit. – hiro protagonist Aug 21 '18 at 12:12

Martijn Pieters · Answer 2 · 2016-12-30T15:41:10.603

List objects do not have awaitable operations, nor do they need to as there is no I/O or other delay you could handle asynchronously.

You also want to use asyncio.sleep(), not time.sleep(); the latter blocks.

The following works just fine; I added in a timestamp in the notify to show this off better:

from datetime import datetime
# ...

start = datetime.now()
def notify():
    length = len(my_list)
    print("t: {:.1f}, List has changed! {}".format(
        (datetime.now() - start).total_seconds(), length))

async def append_task():
    while True:
        await asyncio.sleep(1)
        my_list.append(random.random())
        notify()

async def pop_task():
    while True:
        await asyncio.sleep(1.8)
        my_list.pop()
        notify()

Note that we use await on the asyncio.sleep() call; that provides a point where your coroutine (cooperative routine) yields control to another routine.

This produces:

$ python asyncio_demo.py
t: 1.0, List has changed! 1
t: 1.8, List has changed! 0
t: 2.0, List has changed! 1
t: 3.0, List has changed! 2
t: 3.6, List has changed! 1
t: 4.0, List has changed! 2
t: 5.0, List has changed! 3
t: 5.4, List has changed! 2
t: 6.0, List has changed! 3

Two independent async loops in Python

2 Answers2