How to pass additional parameters to handle_client coroutine?

Question

The recommended way to use asyncio for a socket server is:

import asyncio

async def handle_client(reader, writer):
    request = (await reader.read(100)).decode()
    response = "Data received." 
    writer.write(response.encode())

async def main():
    loop.create_task(asyncio.start_server(handle_client, 'localhost', 15555))

loop = asyncio.get_event_loop()
loop.create_task(main())
loop.run_forever()

This works fine, but now I need to receive appropriate client request and then use aiohttp library to fetch data from a 3rd party restful API.

This requires creating a session variable as follows:

from aiohttp import ClientSession

session = ClientSession()

But this also should be inside a coroutine itself, so I'll put it inside main:

async def main():
    session = ClientSession()
    loop.create_task(asyncio.start_server(handle_client, '', 55555))

Now I need to pass the session variable to the aiohttp get coroutine to fetch the rest API data:

async with session.get(url, params=params) as r:
    try:
        return await r.json(content_type='application/json')
    except aiohttp.client_exceptions.ClientResponseError:
        ....

My question is how can I pass the session variable to handle_client coroutine, if it insists on only having reader,writer parameters, and globals don't help me because sessions must exist inside coroutines?

user4815162342 · Accepted Answer · 2020-06-18T09:39:00.953

21

You can use a temporary function or a lambda:

async def main():
    session = aiohttp.ClientSession()
    await asyncio.start_server(lambda r, w: handle_client(r, w, session),
                               '', 55555)

This works because even though the lambda is not technically a coroutine, it behaves just like one - when invoked it returns a coroutine object.

For larger programs you might prefer a class-based approach where a class encapsulates the state shared by multiple clients without having to pass it explicitly. For example:

class ClientContext:
    def __init__(self):
        self.session = aiohttp.ClientSession()
        # ... add anything else you will need "globally"

    async def handle_client(self, reader, writer):
        # ... here you get reader and writer, but also have
        # session etc as self.session ...

async def main():
    ctx = ClientContext()
    await asyncio.start_server(ctx.handle_client), '', 55555)

edited Jun 18 '20 at 09:39

answered Jun 04 '18 at 14:38

user4815162342

141,790
18
296
355

Thanks for the reply. I can't find any Python documentation verifying that coroutines for lambda function is supported. However, to my amazement, the code actually works when you pass a lambda returning a named coroutine as the parameter to *start_server*! I have my doubts about the second part of your answer though as session variables should only exist inside coroutines... – jsstuball Jun 05 '18 at 05:34
@JSStuball In case of `start_server` it is explicitly documented that its callback can be a plain callback. But in this case the lambda invokes the coroutine (this is a legitimate thing to do - you get the coroutine object) and returns the resulting coroutine object. This makes it functionally equivalent to a coroutine function and it works just fine. You're of course right that the lambda is not a *real* coroutine, for example it cannot contain `await`. – user4815162342 Jun 05 '18 at 06:30
@JSStuball I've fixed the typo in the class example, where `handle_client` should also be a coroutine. – user4815162342 Jun 05 '18 at 06:33
@JSStuball I create it in the `main()` coroutine, just like your code does. The created instance gets **passed** to the `ClientContext` constructor, but that is of no consequence. – user4815162342 Jun 05 '18 at 06:35
Oh yes, my mistake. Right you are. – jsstuball Jun 05 '18 at 06:39
1

@JSStuball I haven't actually checked this, but I suspect that all `ClientSession` cares about is that *some* coroutine is running while it is being created (the code seems to [confirm](https://github.com/aio-libs/aiohttp/blob/e021a01c2f00b058eb3fb3a2d8f3e1f3a89fc2c9/aiohttp/client.py#L122) that). In other words, I figure there would be no warning even if `ClientSession` were instantiated in `ClientContext.__init__`. – user4815162342 Jun 05 '18 at 11:07

How to pass additional parameters to handle_client coroutine?

1 Answers1

Linked