Python - Overloading asynchronous methods

Question

I'm building a wrapper for an API. For now, it works. However, it is synchronous, using the requests module for HTTP calls.

I would like to implement a way to asynchronously call these methods while not having to use a different name or lib version. I immediately thought about overloading, but seeing as overloading in python is a bit different than other languages, it doesn't really look possible.

Essentially, I want to build a class that would look like this (idea-wise, I know that it doesn't work in Python):

class Foo:
    def foo(self):
        # Requests code...
        print("foo sync")

    async def foo(self):
        # aiohttp code...
        print("foo async")

And use it in this way:

f = Foo()
f.foo()
await f.foo()

Output:

>> "foo sync"
>> "foo async"

Essentially, in this code, the async function would just completely override the previous one, which isn't really helpful.

From some googling, it doesn't exactly look possible, however, Python always manages to surprise me.

Thanks in advance :D

Why not just create `foo()` and `foo_async()` methods each dealing with the specific use case? Defining explicit methods for API users is (almost) always better than having a single method perform differently based on its use - for starters, it avoids confusion and doesn't force users to read detailed documentation on how the method might perform for them and their use case. — zwer, Sep 22 '18 at 17:45
@zwer Yeah, so I thought about that, but then I thought that it would be handier to just be able to use overloading style syntax. Now that you say it this way, I think that it would be better to separate the methods. I would have to duplicate most of the code anyway (since async requires async all the way up, AFAIK), and it would be more readable this way. Thanks for the advice. — Plutoberth, Sep 22 '18 at 17:51

score 11 · Accepted Answer · answered Sep 23 '18 at 13:34

Question: Overloading asynchronous methods

Consider the following class definitions, using Inheritance with Overloading!

You have a Base class Foo: from which inherits class Foo_async(Foo):.
Therefore you can reuse most of your implementation, in sync and async mode.
The class Foo_async mainly implements only the async/await requirements.

Note: I'm using asyncio. For example:

import asyncio

class Foo():
    def __init__(self):
        self.mode = "sync"

    def __repr__(self):
        return "{}::foo() {}".format(self.__class__.__name__, self.mode)

    def foo(self):
        print("{}".format(self.__repr__()))
        print("\t\tworkload {}".format(1))
        print("\t\tworkload {}".format(2))

class Foo_async(Foo):
    def __init__(self):
        self.mode = "async awaited"

    async def foo(self):
        super().foo()

Usage:

async def run(foo):
    await foo.foo()

if __name__ == '__main__':
    Foo().foo()

    loop = asyncio.get_event_loop()
    loop.run_until_complete(run(Foo_async()))
    loop.close()

Output:

Foo::foo() sync
        workload 1
        workload 2
Foo_async::foo() async awaited
        workload 1
        workload 2

If you want more granularity, split the workload in def foo(...) into seperate workload functions.
Now you are able to call this functions using await.

For example: (Show, only changed code!)

class Foo():
    ...

    def foo(self):
        print("{}".format(self.__repr__()))
        self.workload_1()
        self.workload_2()

    def workload_1(self):
        print("\t\t{} workload {}".format(self.mode, 1))

    def workload_2(self):
        print("\t\t{} workload {}".format(self.mode, 2))

class Foo_async(Foo):
    ...       

    async def foo(self):
        print("{}".format(self.__repr__()))
        await self.workload_1()
        await self.workload_2()

    async def workload_1(self):
        super().workload_1()

    async def workload_2(self):
        super().workload_2()

Output:

Foo::foo() sync
        sync workload 1
        sync workload 2
Foo_async::foo() async awaited
        async awaited workload 1
        async awaited workload 2

Tested with Python:3.5.3

That's a great idea. I think that what I'll end up doing is splitting my classes into two files: `classes.py` and `classes_async.py`. If they want to use the async versions, they'll have to explicitly import `classes_async` (which will re-implement funcs that should be async), and then just use it with await regularly. So practically what you suggested. That way both files will be relatively thin. Thank you very much, you helped with a big design challenge. — Plutoberth, Sep 23 '18 at 13:42
I get mypy error (Return type "Coroutine[Any, Any, ...]" of "..." incompatible with return type "..." in supertype "..." mypy(error)) when overriding sync methods with their async versions. — erny, Mar 11 '21 at 12:49
great idea , but too much work just to add one more function with different function name and 2 more keywords. — Phyo Arkar Lwin, Oct 02 '21 at 13:51

Python - Overloading asynchronous methods

1 Answers1