Sharing state between two async programs in python using asyncio and contextvars

Question

I currently have two infinite asynchronous tasks running and want to share state between them. One task is a websocket connection that reads in messages then sends messages and the other reads in incoming light data. I want to send a boolean between the two tasks that says whether or not the websocket connection is successful.

Here is how I'm initializing the context.

client_connect_var = contextvars.ContextVar('client_connect',default = False)
client_connect_var.set(False)
ctx = contextvars.copy_context()
async def main():
  message = json.dumps({'payload': {
                        'payload'})
  loop = asyncio.get_event_loop()
  start_light = asyncio.create_task(calculate_idle(3))
  await asyncio.gather(init_connection(message), start_light)

ctx.run(asyncio.run(main()))

Here is my code in my init_connection:

async def init_connection(message):
  async with websockets.connect(uri) as websocket:
      #This should set the global context variable to true
      client_connect_var.set(True)
      CLIENT_WS = websocket
      client_connect = client_connect_var.get()
    # send init message
    await websocket.send(message)
    print("Connection is open")
    while client_connect:
        await handleMessages(websocket, message)
    await websocket.close()

Here is where it's trying to get the current state in the light code

async def calculate_idle(t):
    orig_time = t
    while True:
        await asyncio.sleep(5)
        #This should be true, but it's false
        client_connect = client_connect_var.get()
        await asyncio.sleep(5)

In calculate_idle, the variable in the context is still set to false. I'm not sure how to get it to be set to true from inside init_connection. I'd like to expand this to updating other values in state that are objects. Any help would be appreciated!

Can you reduce the code to the minimum amount required to reflect what is happening? — dirn, Mar 21 '20 at 23:24
Why are you using `ctx = contextvars.copy_context()` and then `ctx.run(asyncio.run(main()))`? — DurandA, Mar 22 '20 at 04:45
I thought I was copying the original context with the variable then running the main inside that copied context. — avenmia, Mar 22 '20 at 14:00

score 5 · Accepted Answer · answered Mar 22 '20 at 06:13

5

ContextVar serves the purpose opposite of what you're trying to use it for: it's for data that is local to a task, something like a "task-local" global variable.

To share data between two tasks you can use normal Python sharing mechanisms, e.g. a global variable or an ordinary mutable value (e.g. a dict or an instance of a class of your choosing). You can also use synchronization like asyncio.Event (https://docs.python.org/3/library/asyncio-sync.html#asyncio.Event) to notify the other task when the value changes.

answered Mar 22 '20 at 06:13

user4815162342

141,790
18
296
355

When I tried using a global variable, I was having the same issue of it not updating. I'm not sure if using asyncio.create_task put it in a different context? It seemed the global variable would be initially set, then get changed in the init_connection, but would still be the initial value in calculate_idle. – avenmia Mar 22 '20 at 14:03
@avenmia Did you update the variable correctly? Have you used `print`s to check that the code was actually being executed? Such things should work, there is nothing magical about asyncio (at that level) that would prohibit such state sharing from working. – user4815162342 Mar 22 '20 at 14:06
I just reverted back to using a global variable and using prints to check that it's being executed and it is. I first, initialize global variable to false, in init_connection I use the global keyword to get the variable. When it is connected I print ("connection set to true") and set the global to true. In the while true in calculate_idle I print the state of the global variable and it remains false. – avenmia Mar 22 '20 at 14:52
Not sure if this is relevant, but I noticed when I print the addresses of the global variable in both spots, they have different addresses. – avenmia Mar 22 '20 at 15:01
@avenmia Can you please show the code that uses the globals? – user4815162342 Mar 22 '20 at 15:16
Yeah would it be easier to ask in a different question with global variables instead? – avenmia Mar 22 '20 at 15:17
1

@avenmia Good point, I guess editing the question and posting a new one are both valid approaches, so it's up to you. It might well turn out that the real problem was elsewhere in the code. – user4815162342 Mar 22 '20 at 15:21
https://stackoverflow.com/questions/60801288/how-to-update-global-variable-in-asyncio-create-task updated question for using a global here.Thank you for your help! – avenmia Mar 22 '20 at 15:36
1

@avenmia Python's global variables are actually module-local; see my answer to the other question. – user4815162342 Mar 22 '20 at 16:08

Sharing state between two async programs in python using asyncio and contextvars

1 Answers1