13

I'm pretty new to the asyncio for python 3.6

So the thing is I have a class and I want to init some property in there. And one of the property is the return value from an async function.

What's the best practice to do this?

  1. Call event_loop one time in the init function to get the return value?

  2. Make the __init__ function async? and run it in the event loop?

Cheers!

UPDATE AGAIN:

Following is my code:

import asyncio
import aioredis
from datetime import datetime

class C:
    def __init__(self):
        self.a = 1
        self.b = 2
        self.r = None
        asyncio.get_event_loop().run_until_complete(self._async_init())

    async def _async_init(self):
        # this is the property I want, which returns from an async function
        self.r = await aioredis.create_redis('redis://localhost:6379')

    async def heart_beat(self):
        while True:
            await self.r.publish('test_channel', datetime.now().__str__())
            await asyncio.sleep(10)

    def run(self):
        asyncio.get_event_loop().run_until_complete(self.heart_beat())

c=C()
c.run()
qichao_he
  • 4,204
  • 4
  • 15
  • 24
  • It kind of depends on why the function is async, how the class is going to be used, etc. I know writing a [mcve] that contains enough to get the relevant ideas across for a nontrivial async program is hard to write, but it’s even harder to answer without one. – abarnert Mar 26 '18 at 14:52
  • @abarnert i just edited the question – qichao_he Mar 26 '18 at 14:59
  • The best practice for making `__init__` async is don't. If you try to instantiate an instance while the event loop is already running it won't work. – dirn Mar 26 '18 at 15:06
  • regarding async __init__: http://as.ynchrono.us/2014/12/asynchronous-object-initialization.html – Jean-Paul Calderone Mar 26 '18 at 15:21
  • @dirn ok, but I tried my code, it's actually working I can call another function of the instance, after I init it. (python 3.6) – qichao_he Mar 26 '18 at 15:31
  • 1
    Right. I didn't say it would never work. I gave an example of when it won't work, an example that @user4815162342 also mentioned in their answer. – dirn Mar 26 '18 at 15:34
  • @dirn ah I see. Now I understand, thanks! – qichao_he Mar 26 '18 at 15:37

1 Answers1

10

Call event_loop one time in the init function to get the return value?

If you spin the event loop during __init__, you won't be able to instantiate C while the event loop is running; asyncio event loops don't nest.

[EDIT: After the second update to the question, it appears that the event loop gets run by a non-static method C.run, so run_until_complete in __init__ will work with the code as written. But that design is limited - for example it doesn't allow constructing another instance of C or of a class like C while the event loop is running.]

Make the __init__ function async? and run it in the event loop?

__init__ cannot be made async without resorting to very ugly hacks. Python's __init__ operates by side effect and must return None, whereas an async def function returns a coroutine object.

To make this work, you have several options:

Async C factory

Create an async function that returns C instances, such as:

async def make_c():
    c = C()
    await c._async_init()
    return c

Such a function can be async without problems, and can await as needed. If you prefer static methods to functions, or if you feel uncomfortable accessing private methods from a function not defined in the class, you can replace make_c() with a C.create().

Async C.r field

You can make the r property asynchronous, simply by storing a Future inside of it:

class C:
    def __init__(self):
        self.a = 1
        self.b = 2
        loop = asyncio.get_event_loop()
        # note: no `await` here: `r` holds an asyncio Task which will
        # be awaited (and its value accessed when ready) as `await c.r`
        self.r = loop.create_task(aioredis.create_redis('redis://localhost:6379'))

This will require every use of c.r to be spelled as await c.r. Whether that is acceptable (or even beneficial) will depend on where and how often it is used elsewhere in the program.

Async C constructor

Although __init__ cannot be made async, this limitation doesn't apply to its low-level cousin __new__. T.__new__ may return any object, including one that is not even an instance of T, a fact we can use to allow it to return a coroutine object:

class C:
    async def __new__(cls):
        self = super().__new__(cls)
        self.a = 1
        self.b = 2
        self.r = await aioredis.create_redis('redis://localhost:6379')
        return self

# usage from another coroutine
async def main():
    c = await C()

# usage from outside the event loop
c = loop.run_until_complete(C())

This last approach is something I wouldn't recommend for production code, unless you have a really good reason to use it.

  • It is an abuse of the constructor mechanism, since it defines a C.__new__ constructor that doesn't bother to return a C instance;
  • Python will notice the above and will refuse to invoke C.__init__ even if you define or inherit its (sync) implementation;
  • Using await C() looks very non-idiomatic, even (or especially) to someone used to asyncio.
user4815162342
  • 141,790
  • 18
  • 296
  • 355
  • Is it really not allowed to return a value from `None`? I honestly may have never tried it, because situations where it could be useful are vastly outnumbered by situations where it would be both pointless and misleading, but I’d be a bit surprised if it was illegal. (I wouldn’t be surprised at linters flagging it, and static analyzers flagging every construction of the class because the return value is ignored inside the construction machinery, etc., just a Python interpreter rejecting it or the ref docs explicitly banning it.) – abarnert Mar 26 '18 at 15:46
  • @abarnert Returning anything non-`None` from `__init__` [raises a `TypeError`](https://docs.python.org/3.7/reference/datamodel.html#object.__init__), as enforced by the default metaclass which implements `.__call__`. One could override this using a custom metaclass, but that'd be a different initialization protocol where `__init__` no longer operated by side effect. – user4815162342 Mar 26 '18 at 15:55
  • A more promising route might be to use `async def __new__(...)`, but it's hard to tell whether this would cause breakage somewhere down the line, for example where Python tests whether `T.__new__` returned a `T` instance and uses that to decide whether to call `T.__init__`, and in other similar places. Overriding `T.__new__` or `MetaT.__call__` to return coroutines/futures is what the answer referred to as "ugly hacks". It can probably be done, but definitely not something to recommend as a first line of thought. – user4815162342 Mar 26 '18 at 15:57
  • Now I’m curious when they added this check. Does it go all the way back to 2.3 or 2.2? Anyway, yeah, returning a coro from `__new__` should work. I did the equivalent with Greg Ewing’s pre-asyncio yield from scheduler example, and it’s painful to work through when debugging your code, but you can make it work. But I still had to manually call `__init__` synchronously before the `yield from`, so it might take another step on top of that to actually solve this problem (as in someone might need more than `obj = await Cls(args)` to use it). I’d have to think about that. – abarnert Mar 26 '18 at 16:32