I want Python to read utf-8 from stdin and write utf-8 to stdout, with optional redirection. This is what my code looks like:
#!/usr/bin/python
# echo.py
import sys
import codecs
so = codecs.getwriter('utf8')(sys.stdout)
si = codecs.getreader('utf8')(sys.stdin)
while True:
t = si.readline()
if len(t) <= 1: break
so.write(t)
so.flush()
so.write('\n')
If I run this with text from a sample utf-8 file, the results are what one expects:
$ python echo.py < sample
my résumé
來去 now
┬ ─ │ └
$
which is OK. When stdin is getting the same sample text entered from a terminal, however, the input lines are not immediately echoed, and a single control-D is not recognized as EOF.
$ python echo.py
my résumé
來去 now
┬ ─ │ └
^D
^CTraceback (most recent call last):
File "./echo.py", line 10, in <module>
t = si.readline()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/codecs.py", line 530, in readline
data = self.read(readsize, firstline=True)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/codecs.py", line 473, in read
newdata = self.stream.read(size)
KeyboardInterrupt
Two consecutive control-D are recognized as EOF, however.
$ python echo.py
my résumé
來去 now
┬ ─ │ └
my résumé
來去 now
┬ ─ │ └
$
The first 3 lines after the command line are typed input; the next 3 are echoed.
If I pipe the same terminal input into stdin, echoing is still not immediate, but on a single control-D, everything finally shows up on stdout.
$ cat - | python echo.py
my résumé
來去 now
┬ ─ │ └
my résumé
來去 now
┬ ─ │ └
$
Any suggestions here? I am running Python 2.7.5 under Mac OS X 10.8.5 (Mountain Lion).