0

When I start a Popen with PIPE stdout, the process doesn't exit untill I press Enter (Windows OS, tested with multiple python and lxc versions) e.g.:

import subprocess as sp

cmd = ['lxc', 'exec', 'remote:container', '--', 'echo', '1']

with sp.Popen(cmd, stdout=sp.PIPE, stderr=sp.PIPE) as proc:
    while True:
        print(proc.poll())

prints None None None... (proc.poll() == None when the process is still running) and when I press Enter it immediately exits.

The process itself does it's job, nothing wrong there, only this exiting problem. It probably lies within the scope of the utility I'm calling (lxc) but is there something I can do to debug this or to circumvent it from the python code?

This call works when it is called during a py.test run, when py.test captures all output, maybe I can do something similar to what it does?

Bob
  • 5,809
  • 5
  • 36
  • 53
  • 1
    The process is inheriting your Python script's stdin. Better to override that if you want reliable/consistent behavior; `stdin=sp.DEVNULL` will work in a new enough Python release. `stdin=open('/dev/null')`, otherwise. – Charles Duffy Aug 01 '19 at 19:11
  • BTW, a `while True:` busy loop is not particularly good practice, when you could use `proc.wait()` or another call that doesn't return until the process exits. What are you trying to accomplish thereby? – Charles Duffy Aug 01 '19 at 19:12
  • @CharlesDuffy that helped, thank u very much for the explanation. Now I pass stdin=DEVNULL when capturing stdout and it works perfectly. while True loop is just for the sake of this stripped down debugging example. – Bob Aug 02 '19 at 18:49
  • Glad that helped; I'll add an answer to the effect. – Charles Duffy Aug 02 '19 at 20:32

1 Answers1

1

If you don't want your process to interact with stdin (a category under which waiting for or responding to input on that handle applies), the best approach to prevent that is to not let it inherit your original Python script's handle (as happens by default with no stdin argument passed to Popen).

A few ways to do that:

  • Add the argument stdin=subprocess.DEVNULL (on a new version of Python)
  • Add the argument stdin=open('/dev/null')
  • Add the argument stdin=subprocess.PIPE, and then call proc.stdin.close(), or proc.communicate('') (of course, only if you only want communicate()'s other effects).
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • 1
    FWIW, `subprocess.DEVNULL` exists since Python 3.3, which is hardly "new"! – ash Aug 02 '19 at 20:53
  • Heh. I'm finally reaching the point of acknowledging Python 3 as something other than a newfangled experiment (it's only been what, 11.5 years?) – Charles Duffy Aug 02 '19 at 21:25