2

I'm excuting a shell command by os.system(). I planned to run it for 1 second, then terminate it if time exceeded. Here's what I tried instead as a test.

import os, time, asyncio

async def cmd():
    os.system("my.py > my.txt") # this processes longer than 1 second

@asyncio.coroutine
def run():
    try:
        yield from asyncio.wait_for(cmd(), 1) # this should excute for more than 1 sec, and hence raise TimeoutError
        print("A")
    except asyncio.TimeoutError:
        os.system("killall python")
        print("B")
    
asyncio.run(run())

But the result was always "A" and the txt was written by my.py.

I also tried:

import os, time, asyncio

async def cmd():
    os.system("my.py > my.txt") # longer than 1 sec

async def run():
    try:
        await asyncio.wait_for(cmd(), 1) # should raise TimeoutError
        print("A")
    except asyncio.TimeoutError:
        os.system("killall python")
        print("B")
    
asyncio.run(run())

results in the same output. What's wrong with the code? I'm pretty new to asyncio and never used it before. Thanks in advance. It's most likely not the problem that wait_for does not stop the cast automatically, since I have a killall python in the second part and in fact, the wait_for function never raises the timeout error, that's the problem.

Lab
  • 196
  • 3
  • 18
  • `cmd` contains two blocking commands. It never yields back control to the event loop which I believe is necessary for the loop to kill your task. Does using `await asyncio.sleep(2)` change the result? It it does, this won't really solve your problem, but it should help point you in the right direction. – dirn Feb 05 '21 at 16:03
  • @dirn What does `yield from` here does? And by the way, the original `cmd()` function contains only 1 line, that is, the `os.system()` line which should run over 1 second. The purpose of `time.sleep(2)` here is just to make sure that the function runs over 1 second. – Lab Feb 05 '21 at 16:08
  • The `yield from` does the same thing as the `await` in your second version. `asyncio.coroutine` has been deprecated in favor of the dedicated `async`/`await` syntax so you should use the second version. – dirn Feb 05 '21 at 16:22
  • I figured your real code didn't include the sleep; that's why I said awaiting the asyncio version wouldn't really solve your problem. If it fixed this specific version of `cmd`, though, it would indicate that using something asyncio-aware instead of `os.system` would potentially accomplish what you want. – dirn Feb 05 '21 at 16:22
  • @dirn So setting a timeout for asyncio.wait_for doesn't work here? – Lab Feb 06 '21 at 06:58
  • 1
    Probably not to stop `os.system`. You might have luck with `asyncio.create_subprocess_exec`, but I’m not sure if that will kill the subprocess or just the task that’s controlling it. – dirn Feb 06 '21 at 13:11
  • @dirn I mean, os.system("killall python") will probably do the job, but the problem here is that it will always print "A", not to even process the `except` part. – Lab Feb 06 '21 at 13:32
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/228350/discussion-between-acc-lab-and-dirn). – Lab Feb 06 '21 at 13:38

1 Answers1

2

os.system is a blocking operation, so it doesn't play well with asyncio. Whenever you use asyncio, all "longer" operations need to be asynchronous, or you lose all benefits of going with asyncio in the first place.

import asyncio

async def cmd():
    try:
        proc = await asyncio.create_subprocess_shell(
            "sleep 2 && date > my.txt",
            shell=True,
            stdout=asyncio.subprocess.DEVNULL,
            stderr=asyncio.subprocess.DEVNULL)
        await proc.communicate()

    except asyncio.CancelledError:
        proc.terminate()
        print("X")

async def run():
    try:
        await asyncio.wait_for(cmd(), 1)
        print("A")

    except asyncio.TimeoutError:
        print("B")

asyncio.run(run())

As @dim mentions in comments, asyncio.create_subprocess_exec will launch an external command asynchronously. wait_for will actually raise two exceptions: TimeoutError in the awaiting code, and CancelledError in the awaited code. We can use the latter to realise the operation was cancelled, and to terminate or kill our subprocess (as this is not done automatically).

Amadan
  • 191,408
  • 23
  • 240
  • 301
  • The code is giving me `NotImplementedError` – Lab Feb 08 '21 at 05:08
  • The code I posted runs correctly as-is on at least python3.7 and python3.8 under Mac OS X (and almost certainly Linux as well). If you are having issues with it, please report the Python version as well as the exact line where the error happens, and I will look into it. – Amadan Feb 08 '21 at 05:14
  • Which python version, which OS, which line of my code fits in one line though. – Amadan Feb 08 '21 at 07:36
  • File "C:\Users\thomas\Desktop\a.py", line 24, in asyncio.run(run()) / return loop.run_until_complete(main) / return future.result() / await asyncio.wait_for(cmd(), 1) / return fut.result() / stderr=asyncio.subprocess.DEVNULL) / stderr=stderr, **kwds) / protocol, cmd, True, stdin, stdout, stderr, bufsize, **kwargs) / The last line: File "C:\Users\thomas\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 458, in _make_subprocess_transport raise NotImplementedError NotImplementedError – Lab Feb 08 '21 at 10:56
  • 1
    So, Windows. I am not an expert on Windows programming, but I am almost certain this is the source (and possibly solution) of the problem: [Why am I getting NotImplementedError with async and await on Windows?](https://stackoverflow.com/questions/44633458/why-am-i-getting-notimplementederror-with-async-and-await-on-windows) – Amadan Feb 08 '21 at 14:11