Is it possible to have race condition in python aiohttp socketio?

Question

I am working on code similar to below code. Sometimes the program stops working or I get strange errors regarding socketio session access. Slowly I feel it could be race conditions.

Its more pseudo code. I want to demonstrate, that I access global shared state and the socketio sessions from multiple coroutines.

import asyncio as aio
from aiohttp import web
import socketio


app = web.Application()
sio = socketio.AsyncServer()

app["sockets"] = []

@sio.on("connect")
async def connect(sid):
    app["sockets"].append(sid)

@sio.on("disconnect")
async def disconnect(sid):
    app["sockets"].remove(sid)

@sio.on("set session")
async def set_session(sid, arg):
    await sio.save_session(sid, {"arg": arg})

async def session_route(req):
    data = await req.json()
    for sid in app["sockets"]:
        await sio.save_session(sid, {"arg": data["arg"]})
    return web.Response(status=200)

if __name__ == '__main__':
    web.run_app(app)

score 2 · Accepted Answer · answered Sep 23 '19 at 06:49

There is definitely a problem here:

for sid in app["sockets"]:  # you are iterating over a list here
    await sio.save_session(...)  # your coroutine will yield here

You are iterating over the list app["sockets"] and in each iteration you use the await keyword. When the await keyword is used, your coroutine is supended and the event loops checks if another coroutine can be executed or resumed.

Let's say the connect(...) coroutine is run while session_route is waiting.

app["sockets"].append(sid)  # this changed the structure of the list

connect(...) changed the structure of the list. This can invalidate all iterators that currently exist for that list. The same goes for the disconnect(...) coroutine.

So either don't modify the list or at least don't reuse the iterator after the list has changed. The latter solution is easier to achieve here:

for sid in list(app["sockets"]):
    await sio.save_session(...)

Now the for-loop iterates over a copy of the original list. Changing the list now will not "disturb" the copy.

Note however, that additions and deletions from the list are not recognized by the copy.

So, in short, the answer to your question is yes, but it has nothing to do with async io. The same issue can easily occur in synchronous code:

for i in my_list:
    my_list.remove(1)  # don't do this

Another option would be to protect the list of sockets with a `asyncio.Lock` so that the loop over the sockets and the addition and removal of sockets can not be interweaved. All adds and removes would wait until after the the save loop completed. The issue that you point out at the end would result in a deadlock if a lock was taken on the list and remove required that lock. — Dan D., Sep 23 '19 at 07:16

Is it possible to have race condition in python aiohttp socketio?

1 Answers1