How to Create Custom Events for Python Asyncio

Question

The Mesos scheduler Marathon has an asynchronous HTTP API. E.g. when one deploys an app by posting a JSON to /v2/apps a deployment id is returned. The id can then be used to either poll the deployment state in /v2/deployments or by subscribing to /v2/events and look for the deployment_success event.

I would like to create an asynchronous Python client with coroutines. E.g. client.deploy_app(...) should return once the deployment_success event arrived but not block.

How can I implement these methods with asyncio? How can I create an event listener? It feels an event loop is made for this but I don't see how I register events.

score 2 · Answer 1 · answered Dec 16 '17 at 09:37

Creating asynchronous post http requests that needed for /v2/apps can be done with aiohttp module:

import asyncio
import aiohttp


async def post(url, json):
    async with aiohttp.ClientSession() as session:
        async with session.post(url, json=json) as resp:
            return await resp.json()


async def main():
    res = await post('http://httpbin.org/post', {'test': 'object'})
    print(res['json'])  # {'test': 'object'}


loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(main())
    loop.run_until_complete(loop.shutdown_asyncgens())
finally:
    loop.close()

If you want to use /v2/events to track deployment success you should request for stream (see api doc). It can be achieved in aiohttp with it's asynchronous iteration: you just asynchronously read content line-by-line waiting for event you need, for example:

import asyncio
import aiohttp


async def stream(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as resp:
            async for line in resp.content:
                yield line


async def main():
    async for line in stream('http://httpbin.org/stream/10'):
        print(line)  # check if line contains event you need


loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(main())
    loop.run_until_complete(loop.shutdown_asyncgens())
finally:
    loop.close()

If you want to use /v2/deployments you should request it periodically waiting some delay using asyncio.sleep. In this case your function won't be blocking:

import asyncio
import aiohttp


async def get(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as resp:
            return await resp.json()


async def main():
    while True:       
        # Make request to see if deplayment finished:
        res = await get('http://httpbin.org/get')
        print(res['origin'])  # break if ok here
        # Async wait before next try:
        await asyncio.sleep(3)


loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(main())
    loop.run_until_complete(loop.shutdown_asyncgens())
finally:
    loop.close()

Thanks for your help. I'm afraid I did not express my question clear enough. I'd like to know whether there is another way than `async def deploy(app): deployment_id = await post(app) await deployment_event(deployment_id)` which has a race condition anyways. — Karsten, Dec 18 '17 at 09:22

How to Create Custom Events for Python Asyncio

1 Answers1