aiohttp - How to save a persistent session in class namespace

Question

I am trying to use aiohttp in one of my projects and struggling to figure out how to create a persistent aiohttp.ClientSession object. I have gone through the official aiohttp documentation but did not find it help in this context.

I have looked through other online forums and noticed that a lot has changed ever since aiohttp was created. In some examples on github, the aiohttp author is shown to be creating a ClientSession outside a coroutine functions (i.e. class Session: def __init__(self): self.session = aiohttp.ClientSession()). I also found that one should not create a ClientSession outside coroutine.

I have tried the following:

class Session:
    def __init__(self):
        self._session = None

    async def create_session(self):
        self._session = aiohttp.ClientSession()

    async fetch(self, url):
        if self._session is None:
            await self.create_session()
        async with self._session.get(url) as resp:
            return await resp.text()

I am getting a lot of warning about UnclosedSession and connector. I also frequently get SSLError. I also noticed that 2 out of three calls gets hung and I have to CTRL+C to kill it.

With requests I can simply initialize the session object in __init__, but it's not as simple as this with aiohttp.

I do not see any issues if I use the following (which is what I see as example all over the place) but unfortunately here I end up creating ClientSession with every request.

def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as resp:
            return await resp.text()

I can wrap aiohttp.ClientSession() in another function and use that as context-manager, but then too I would end up creating a new session object every time I call the wrapper function. I am trying to figure how to save a aiohttp.ClientSession in class namespace and reuse it.

Any help would be greatly appreciated.

This might help https://stackoverflow.com/questions/54770360/how-can-i-wait-for-an-objects-del-to-finish-before-the-async-loop-closes — antfuentes87, Mar 26 '19 at 02:26

Yurii Kramarenko · Answer 1 · 2019-03-27T10:37:45.017

2

Here is working example:

from aiohttp import ClientSession, TCPConnector
import asyncio


class CS:

    _cs: ClientSession

    def __init__(self):
        self._cs = ClientSession(connector=TCPConnector(verify_ssl=False))

    async def get(self, url):
        async with self._cs.get(url) as resp:
            return await resp.text()

    async def close(self):
        await self._cs.close()


async def func():
    cs = CS()
    print(await cs.get('https://google.com'))
    await cs.close()  # you must close session


loop = asyncio.get_event_loop()
loop.run_until_complete(func())

edited Mar 27 '19 at 10:37

answered Mar 26 '19 at 10:39

Yurii Kramarenko

1,025
6
16

2

Which aiohttp version did you test with? I'd tried the same code (without explicitly passing the `TCPConnector`) but I was getting `RuntimeError: Timeout context manager should be used inside a task`. Moreover, creating `ClientSession` outside coroutine is considered dangerous and is not recommended. – user6037143 Mar 26 '19 at 10:55
@user6037143 I'm using last aiohttp version (3.5.4, python 3.7) – Yurii Kramarenko Mar 26 '19 at 15:33
I have the same setup, but I get `RuntimeError: Timeout context manager should be used inside a task` when I `get` method. Also, the docs and faqs mention that creation of `ClientSession` outside coroutine is dangerous – user6037143 Mar 26 '19 at 16:46
If I call `asyncio.run(func())`, I get `RuntimeError: Timeout context manager should be used inside a task`. If I use `loop = asyncio.get_event_loop(); loop.run_until_complete(func())` it works. – user6037143 Mar 26 '19 at 16:51
@user6037143 example has been changed – Yurii Kramarenko Mar 27 '19 at 17:58

score 0 · Answer 2 · answered Oct 18 '21 at 02:59

You can do it.

I implemented a way to share session when writing Django programs (using asgi).Use pid to mark the session of different processes, which is convenient for django to call between different processes.

After actual testing, I can directly call the shared session.

Django 3.2
uvicorn

aiohttp.py

import os
import asyncio
import aiohttp
import logging

session_list = {}
logger = logging.getLogger(__name__)


class Req:

    @property
    def set_session(self):
        try:
            loop = asyncio.get_running_loop()
        except:
            loop = asyncio.get_event_loop()
            asyncio.set_event_loop(loop)
        session = aiohttp.ClientSession(loop=loop)
        session_list.update({os.getpid(): session})
        return session

    def __init__(self):
        if session_list.get(os.getpid()):
            self.session = session_list.get(os.getpid())
        else:
            self.session = self.set_session

    async def test(self):
        if session_list:
            session = session_list.get(os.getpid())
            if session and session.closed:
                session_list.pop(os.getpid())
                session = self.set_session
        else:
            session = self.set_session

        if not session or session.loop.is_running():
            session = self.set_session
            logger.warning("session abnormal")
        result = await session.get("http://httpbing.org/get")
        print(result.status)


req = Req()

views.py

from django.http import HttpResponse
from django.shortcuts import render  # noqa
from django.views.generic import View
from django.utils.decorators import classonlymethod

import asyncio


class TTT(View):

    @classonlymethod
    def as_view(cls, **initkwargs):
        view = super().as_view(**initkwargs)
        view._is_coroutine = asyncio.coroutines._is_coroutine
        return view

    async def get(self, request):
        await req.test()
        return HttpResponse("ok")

aiohttp - How to save a persistent session in class namespace

2 Answers2