Is there a way to await on an asyncio.Task.result from sync code

Question

I've a situation like below,

event_loop = asyncio.new_event_loop()

async def second_async():
    # some async job
    print("I'm here")
    return 123

def sync():
    return asyncio.run_coroutine_threadsafe(second_async(), loop=event_loop).result()

async def first_async():
    sync()


event_loop.run_until_complete(first_async())

I call the sync function from a different thread(where the event_loop is not running), it works fine. The problem is if I run the event_loop.run_complete... line, the .result() call on the Task returned by run_coroutine_threadsafe blocks the execution of the loop, which makes sense. To avoid this, I tried changing this as follows,

event_loop = asyncio.new_event_loop()

async def second_async():
    # some async job
    print("I'm here")
    return 123


def sync():
    # if event_loop is running on current thread
        res = loop.create_task(second_async()).result()
    # else
        res = asyncio.run_coroutine_threadsafe(second_async(), loop=event_loop).result()
    
    # Additional processing on res
    # Need to evaluate the result of task right here in sync.
    return res

async def first_async():
    sync()


event_loop.run_until_complete(first_async())

This works fine, but the .result() call on the Task object returned by create_task always raises an InvalidStateError. The set_result is never called on the Task object.

Basically, I want the flow to be as such (async code) -> sync code (a non blocking call ->) async code

I know this is a bad way of doing things, but I'm integrating stuff, so I don't really have an option.

Answer 1

Here is a little single-threaded program that illustrates the problem.

If you un-comment the line asyncio.run(first_async1()), you see the same error as you're seeing, and for the same reason. You're trying to access the result of a task without awaiting it first.

import asyncio
    
event_loop = asyncio.new_event_loop()

async def second_async():
    # some async job
    print("I'm here")
    return 123

def sync1():
    return asyncio.create_task(second_async()).result()

async def first_async1():
    print(sync1())

def sync2():
    return asyncio.create_task(second_async())

async def first_async2():
    print(await sync2())
    
# This prints I'm here,
# the raises invalid state error: 
# asyncio.run(first_async1())

# This works, prints "I'm here" and "123"
asyncio.run(first_async2())

With that line commented out again, the second version of the program (first_async2) runs just fine. The only difference is that the ordinary function, sync2, returns an awaitable instead of a result. The await is done in the async function that called it.

I don't see why this is a bad practice. To me, it seems like there are situations where it's absolutely necessary.

Another approach is to create a second daemon thread and set up an event loop there. Coroutines can be executed in this second thread with asyncio.run_coroutine_threadsafe, which returns a concurrent.futures.Future. Its result method will block until the Future's value is set by the other thread.

#! python3.8

import asyncio
import threading

def a_second_thread(loop):
    asyncio.set_event_loop(loop)
    loop.run_forever()
    
loop2 = asyncio.new_event_loop()
threading.Thread(target=a_second_thread, args=(loop2,), daemon=True).start()

async def second_async():
    # some async job
    print("I'm here")
    for _ in range(4):
        await asyncio.sleep(0.25)
    print("I'm done")
    return 123

def sync1():
    # Run the coroutine in the second thread -> get a concurrent.futures.Future
    fut = asyncio.run_coroutine_threadsafe(second_async(), loop2)
    return fut.result()

async def first_async1():
    print(sync1())

def sync2():
    return asyncio.create_task(second_async())

async def first_async2():
    print(await sync2())
    
# This works, prints "I'm here", "I'm done", and "123"
asyncio.run(first_async1())

# This works, prints "I'm here", "I'm done", and "123"
asyncio.run(first_async2())

Of course this will still block the event loop in the main thread until fut.result() returns. There is no avoiding that. But the program runs.