Python subprocess output stream stdout.read hangs forever for input prompt

Question

I am trying to run a subprocess with Popen, and at a time it asks for an input prompt without any EOF, so stdout.read() blocks the while loop until an EOF is found, like forever because there will be none.

I am unable to detect if we are in an input prompt coming next via

• proc.stdout.isatty() it stays at False
• proc.stdout.writable() it stays at False

main1.py

from subprocess import Popen, PIPE
import sys

proc = Popen(["python3", "main2.py"], stdin=PIPE, stdout=PIPE, stderr=PIPE)
def read_until_it_finishes():
    while proc.poll() is None:
        if proc.stdout.isatty() is True: # <-- Why this line isn't it detecting we are in a input prompt ?
            break
        if proc.stdout.writable() is True: # <-- Why this line isn't it detecting we are in a input prompt ?
            break
        line =  proc.stdout.read(1).decode(sys.stdout.encoding) # https://stackoverflow.com/a/63137801/10294022
        sys.stdout.write(line)
        sys.stdout.flush()

read_until_it_finishes()
proc.stdin.write(b"My name is Catwoman\n")
proc.stdin.flush()

main2.py

import sys

sys.stdout.write("Hello my name is Batman\n")
sys.stdout.flush()
sys.stdout.write("I am awesome\n")
sys.stdout.flush()
name = input('And you what is your name:')
sys.stdout.write(name)
sys.stdout.flush()
sys.stdout.close()

Then run

python3 main1.py

Would you please help me ?

Answer 1

Here is an example how to pass stdin and print stdout between applications:

• runner.py:

from subprocess import Popen, PIPE
import sys

proc = Popen(["python", "app.py"], stdin=PIPE, stdout=PIPE, stderr=PIPE)
def read_until_it_finishes():
    while proc.poll() is None:
        char = proc.stdout.read(1).decode(sys.stdout.encoding)
        sys.stdout.write(char)
        sys.stdout.flush()
        if char == '>':
            name = input('')
            proc.stdin.write(name.encode(sys.stdout.encoding))
            proc.stdin.write(b'\n')
            proc.stdin.flush()

read_until_it_finishes()

• app.py

print('Hello, I`m a bot')

while True:
    name = input('What is your name (or print exit to exit)>')
    if name == 'exit':
        print('bye bye')
        break
    print(f'Hello {name}')

• output:

Hello, I`m a bot
What is your name (or print exit to exit)>a
Hello a
What is your name (or print exit to exit)>b
Hello b
What is your name (or print exit to exit)>c
Hello c
What is your name (or print exit to exit)>exit
bye bye

---

Extra question:

you are using a special character char == '>' to break the loop, in order to avoid using stdout.read again. Actually the app as subprocess I am running doesn't use this kind of character, and I can't edit this app

Extra answer:

If you don't able to expect command line prompt line and don't know specific prompt character that can be used as break point, you may try to switch between input and output by using Timer:

from subprocess import Popen, PIPE
from threading import Timer
import sys

SECONDS = 3

proc = Popen(["python", "app.py"], stdin=PIPE, stdout=PIPE, stderr=PIPE)

def run_input():
    if proc.poll() is None:
        name = input('')
        proc.stdin.write(name.encode(sys.stdout.encoding))
        proc.stdin.write(b'\n')
        proc.stdin.flush()

def read_until_it_finishes():
    timer = Timer(SECONDS, run_input)
    timer.start()
    while proc.poll() is None:
        try:
            char = proc.stdout.read(1).decode(sys.stdout.encoding)
            sys.stdout.write(char)
            sys.stdout.flush()
        finally:
            timer.cancel()
            timer = Timer(SECONDS, run_input)
            timer.start()

read_until_it_finishes()

In this case runner.py will be waiting for input every 3 seconds (SECONDS variable). But this logic is ineffective because:

1. Long latency between inputs. Even if bot is answering in 1 second you have to wait for 3 seconds until to be able do input
2. If bot is answering longer than 3 seconds the answer print will be stopped at the middle until next input

But here is no other way (at least I can't see other way) to determine if app.py is waiting for input or just busy and running some background operations.