Getting the output unbuffered from subprocess is not working Python

Question

I have an exectuable that if i run it with flags it starts a tool that generate messages all the time. When i try to run it with subprocess i get the output to the cmd and that is fine. The thing is that i want to take this messages and do something with them but for some reason the unbuffring method is not working and I'm not getting any message .

The following is the code that I tried :

p = subprocess.Popen(
 ["my_tool.exe","runMode","y","port","1234"],stdout=subprocess.PIPE,stderr=subprocess.STDOUT, bufsize=1)

To generate the output tried the folloing:

for line in p.stdout:
    print("CUR", line)

and:

while p.poll() is None:
    line = p.stdout.read(2)
    print("Print:" + line)

and:

for line in iter(p.stdout.readline, b''):
    print(line)
p.stdout.close()

Any idea why the code blocks on the stdout line alwayes ?

UPDATE:

For example if i have the following infinite program:

inf.py

import time
i = 0
while True:
    time.sleep(0.5)
    print(i)
    i += 1

and the following code that runs it:

p = subprocess.Popen(r"C:\Python35\python.exe inf.py",
                     stdout=subprocess.PIPE, bufsize=1, universal_newlines=True)

and to generate the output I'm using one of the above methods , If i stop it using CTRL+ C it is alwayes stuck on this line:

  File "C:\Python35\lib\encodings\cp1252.py", line 22, in decode
    def decode(self, input, final=False):

Perhaps this is not related, but why are you using `read(2)`? That reads only two bytes at a time. — John Gordon, Jun 03 '18 at 16:13
When i cancel the operation by CTRL+C i get the following line: Traceback (most recent call last): File "lg.py", line 22, in for line in p.stdout: File "C:\Python35\lib\encodings\cp1252.py", line 22, in decode def decode(self, input, final=False): KeyboardInterrupt — KaramJaber, Jun 04 '18 at 06:54

score 0 · Answer 1 · answered May 06 '19 at 15:39

First I set the following environment variable: PYTHONUNBUFFERED=1

Then I changed your code a little:

read_infinite.py

import subprocess

binary = "C:\\Python35\\python.exe"
script = "C:\\temp\\python\\tests\\inf.py"

p = subprocess.Popen([binary, script], stdout=subprocess.PIPE, bufsize=1, universal_newlines=True)

while p.poll() is None:
    #line = p.stdout.read(2)
    line = p.stdout.readline()
    if line:
        print("Result: %r" % line)

Removing bufsize=1 works same.

The error I got at first was that the inf.py was not found, but this was not shown, because the prints of read_infinite.py scrolled the error out. And the process stopped after a lot of lines of "Result:" (variable "line" is empty).

Removing if line: and adding time.sleep(0.5) after the print (remember import time!) shows the same results.

So the solution is: the parent has to wait for the child.

Getting the output unbuffered from subprocess is not working Python

1 Answers1