0

I am subscribing to multiple socket.io servers at the same time, all incoming entries get checked whether they have been already processed/received by another websocket client. The servers provide me with data that should be added into a single list, the order of the list is important, as it represents an orderbook from an exchange. An entry should not be added twice. With my current solution I am experiencing race conditions, as far as I know socket.io is using asyncio, a simple threading.Lock won't help.

sio = socketio.Client()
sio1 = socketio.Client()
locker = threading.Lock()
added_Orders = collections.deque(maxlen=40)

@sio.on('connect')
def connect():
    print('connection established')

@sio1.on('connect')
def connect():
    print('connection1 established')

@sio.on('add_order', namespace="/market")
def add_order(data):
    with locker:
        if data in added_Orders:
            return
        else:
            added_Orders.append(data)
    webRecv.process_add(data)

@sio1.on('add_order', namespace="/market")
def add_order1(data):
    with locker:
        if data in added_Orders:
            return
        else:
            added_Orders.append(data)
    webRecv.process_add(data)
Userqhl
  • 1
  • 2
  • You need to explain better what you mean by "socket.io is using asyncio". The code that you posted does not appear to be using asyncio, so where is asyncio used? – Miguel Grinberg Mar 04 '22 at 10:31
  • @Miguel Thanks for your response! I am not using asyncio in my code, however I was assuming that socketio uses asyncio, since the 'with locker' intended blocks do not lead to the desired behaviour. Sometimes the code is processing duplicate data. – Userqhl Mar 04 '22 at 12:42
  • The `socketio.Client` class does not use asyncio, it uses standard multithreaded APIs in Python from the `threading` package. How complex is your `data` elements? If they are dicts, they'll have to be 100% identical to be detected as duplicates. I think a better strategy in that case would be for you to check some sort of identifier that is a simpler value and not a dict. – Miguel Grinberg Mar 04 '22 at 19:35
  • @Miguel thank you so much, what a simple solution! Indeed they are dicts, apperently there is a value different to the data received by the other clients. Extracting the id and comparing it works! – Userqhl Mar 05 '22 at 00:02

0 Answers0