0

I have an asyncio project. It has several modules. Many of them need access to a few globals, like: 1. aiohttp ClientSession() object, since according to aiohttp docs, I should avoid creating a new ClientSession for every request.
2. asyncio socket, i.e reader, writer which I create using asyncio.open_connection(). I want to maintain a persistent connection.
3. event loop, which I get using a asyncio.get_event_loop()

What is the best practice to share such variables?
I would like to create a globals.py module, which will define these variables.

The problem is that I can't use the async with syntax for the ClientSession object in the globals module.
For the socket, I must define it somehow in an async def, so I can't expose it at the module level.

And, testing wise - should every module define a global variable like:
loop = asyncio.get_event_loop()
Or, is it better to pass the event loop to the module, for example in the class __init__)?

user3599803
  • 6,435
  • 17
  • 69
  • 130

1 Answers1

2

There is no need to use globals. Instead, create a class that stores the "global" data and make your functions methods on the class. For example:

class Operation:
    def __init__(self):
        self._http = aiohttp.ClientSession()

    async def open(self):
        self._reader, self._writer = \
            await asyncio.open_connection(<host>, <port>)

    # ... more methods ...

    async def close(self):
        await self._http.close()
        self._writer.close()
        await self._writer.wait_closed()

This has several advantages:

  • It doesn't require any global state;
  • You have full control where the state is constructed and torn down (all at once);
  • If you decide you need more than one global operation done in parallel, it is easy to do so - just create more than one instance of Operation.
  • When testing, you can simply create and tear down Operation instances as needed.

With that in place, your main coroutine can look like this:

async def main():
    op = Operation()
    await op.open()
    try:
        await op.do_something()
        ...
    finally:
        await op.close()

asyncio.run(main())
#or asyncio.get_event_loop().run_until_complete(main())

Note that the event loop is not stored on the object, or passed to it in any way. This is because the event loop can always be obtained with asyncio.get_event_loop() which, when called from a coroutine, is guaranteed to return the loop that is currently running.

user4815162342
  • 141,790
  • 18
  • 296
  • 355
  • I still dont see how to share these across modules. If a module need access to the reader, writer. The main should pass it the operation object? Same if a module needs the http session – user3599803 Dec 20 '18 at 08:06
  • @user3599803 You would share those just like anything else in python. The code in a module is typically organized in functions and classes. The classes could receive an operation (feel free to choose a better name) only once, in `__init__`, and module-level functions can simply accept the arguments they actually need (not necessarily the whole operation). One of the drawbacks of global state is that you never know who is using what. – user4815162342 Dec 20 '18 at 09:37
  • And how is it different from module global variables? Because such a class should be a singelton in my project. Also I can't see the async with usage with the clientsession - I see it is impossible to use in this case? – user3599803 Dec 20 '18 at 18:30
  • The point is that it doesn't *need* to be a singleton (in the sense of the singleton pattern), you choose it to be such. Just like `aiohttp.ClientSession`: it could be implemented as a global, but it gives you more control for it not to be. As for `async with`, that shouldn't matter as long as you correctly close the session. – user4815162342 Dec 20 '18 at 21:03