How to continuously read from stdin (not just once input file is done)?

Question

I have these two scripts:

clock.py

#!/usr/bin/env python3
import time

while True: 
    print("True", flush=True)
    time.sleep(1)

continuous_wc.py

#!/usr/bin/env python3
import sys

def main():
    count = 0
    for line in sys.stdin:
        sys.stdout.write(str(count))
        count += 1

if __name__=='__main__':
    main()

And I run them like so:

./clock.py | ./continuous_wc.py

I'm hoping that it prints:

1
2
3
4
5
...

Every second is like a clock, because it's counting the lines in the file basically. But it doesn't output anything. Why not?

The problem isn't with reading, it's with writing to a pipe. — Barmar, Jan 12 '22 at 21:53
@Barmar thank you. I have updated the question with that suggestion. It still seems not to have any output. — BigBoy1337, Jan 12 '22 at 22:00
You should write a newline after the count in `continuous_wc.py`, since you want each number on a different line. That will also solve the problem, since output is line-buffered. — Barmar, Jan 12 '22 at 22:01
Why are you using `sys.stdin` and `sys.stdout`, instead of `input()` and `print()`? — Barmar, Jan 12 '22 at 22:02
I think what Barmar is suggesting is that you also need to flush the output from `continuous_wc.py`. — Code-Apprentice, Jan 12 '22 at 22:05
You could try `export PYTHONUNBUFFERED=1 ; ./clock.py | ./continuous_wc.py` — Mark Setchell, Jan 12 '22 at 22:06
If your Python is 3.6 or newer, you can write more elegantly (and with the desired newline): `sys.stdout.write(f"{count}\n")`. — Mike Clark, Jan 12 '22 at 22:24

Mike Clark · Accepted Answer · 2022-01-13T16:18:36.797

4

In addition to print(x, flush=True) you must also flush after sys.stdout.write.

Note that the programs would technically work without flush, but they would print values very infrequently, in very large chunks, as the Python IO buffer is many kilobytes. Flushing is there to make it work more real-time.

sys.stdout.write(str(count))
sys.stdout.flush()

edited Jan 13 '22 at 16:18

answered Jan 12 '22 at 22:07

Mike Clark

10,027
3
40
54

score 3 · Answer 2 · edited Jan 13 '22 at 11:53

Using everyone's advice, this is what I have now (which works):

clock.py

#!/usr/bin/env python3
import time

while True:
    print("True", flush=True) 
    time.sleep(1)

continuous_wc.py

#!/usr/bin/env python3
import sys

def main():
    count = 0
    for line in sys.stdin:
        print(count, flush=True, end='\r')
        count += 1

if __name__=='__main__':
    main()

I had to use flush=True in both the write and the read scripts. However when using PYTHONUNBUFFERED=1, I was able to skip this and it worked. Thanks @Mark Setchell. Additionally, I am using print and stdin but it seems fileinput.input works as well.

How to continuously read from stdin (not just once input file is done)?

2 Answers2