-1

Let's say I have the following program

import requests

async def foo():
    res = await get_resource()
    status_code = await make_request(res)

    print(f"status code: {status_code}\n", res)

async def get_resource():
    # ...
    with open('file') as f:
        return f.read()

async def make_request(data):
    return requests.post('/create', data=data).status_code

This program gains nothing from being async and can hamper the performance of other code on the event loop. Obviously, this is a simple example, but across multiple files and developers, it may not be so easy to spot.

It would be awesome if I could add something to my build pipeline that could detect this case. Is there any Python build tool that will indicate that this async code is requiesting synchronous I/O?

ollien
  • 4,418
  • 9
  • 35
  • 58
  • “ can hamper the performance of other code” based on what evidence? I doubt the overhead is worth it – Tadhg McDonald-Jensen Jul 17 '20 at 20:44
  • My understanding is that by calling synchronous code, other async code will never be yielded to. Is that incorrect? Regardless, in the codebase I am working in, we're trying to segregate this pathway. – ollien Jul 17 '20 at 20:57
  • Oh I misread, you want to change the operations to async not change functions to non async if they aren’t using async stuff. Yes that will limit performance but how would you expect to identify synchronous operations that have async equivalents? Like file io only has 3rd party options for async so any build tool that identifies it would have to recommend a specific library which makes it less universal a tool. – Tadhg McDonald-Jensen Jul 17 '20 at 21:55
  • Yeah, I was hoping it would be able to analyze those statically. Might be a bit of a pipe dream, though. – ollien Jul 18 '20 at 01:20
  • well I mean like writing code that checks for using the `open` function inside an `async` function wouldn't be too hard, it's just that coming up with a solid set of rules for which functions should be replaced and what they need to be replaced with I can't possibly see a way other than defining it manually. – Tadhg McDonald-Jensen Jul 18 '20 at 02:59

1 Answers1

2

You could use asyncio debug mode. Among other things, it shows log message if a function takes longer than 100ms (may be tuned):

WARNING: Executing <Task finished coro=<inner() done, defined at
asyncio_debug.py:33> result=None created at asyncio_debug.py:43>
took 0.101 seconds
alex_noname
  • 26,459
  • 5
  • 69
  • 86
  • Sadly, not feasible, and doesn't necessarily solve the problem. This would require profiling test runs, which may not catch these cases. Static analysis would be preferred. – ollien Jul 18 '20 at 01:20