12

I needed to switch the standard input to non-buffered mode in Python, so that I can read single characters off it. I managed to get it working, but now the standard output is broken: somehow it seems like after the newline character, some space characters are emitted, zero on the first line, 3 on the second, 6 on the third, etc, like this:

ASD
   ASD
      ASD

Operating system is Ubuntu Linux 12.04, 64-bit edition, Python version is 3.2.3.

How can I rid myself from this behavior?

Below is the code I've used:

import sys
import tty
import termios

fd = sys.stdin.fileno()
old_settings = termios.tcgetattr(fd)
tty.setraw(sys.stdin)

for i in range(0, 10):
    print("ASD")

termios.tcsetattr(fd, termios.TCSADRAIN, old_settings)
K.Steff
  • 2,164
  • 3
  • 19
  • 25
  • Looks like you switched `fd` and `sys.stdin` arguments. `termios.tcgetattr` takes something that thas `fileno` attribute and `tty.setraw` awaits file descriptor (at least in python3.9). What version are you using? – Adam Jenča Jul 05 '21 at 12:05

3 Answers3

19

The problem you're experiencing is the difference between 'raw', 'cooked', and 'cbreak' modes. And these modes are modes of the kernel level terminal driver, not modes of your application code or the standard library or anything else in userspace. This is the old-school Unix way of referring to these. Posix has replaced them with a much more fine-grained set of attributes, though the Posix attributes are typically flipped in concert with helper functions in a way that mimics the old 'raw', 'cooked' and 'cbreak' modes.

In cooked mode, the terminal driver itself has primitive line editing functionality builtin. It handles backspace, word erase (basically backspace a whole word at once) and similar things. Nothing sophisticated like handling arrow keys or history or anything like that. Very primitive. In this mode, your program never sees anything from the terminal until the end-of-line (eol) character is sent, and then your program gets a whole line, and the line ending is translated to Unix standard \n regardless of what the terminal actually does. Also, as part of this, the terminal driver echoes typed characters back to the terminal so the user can see what they're typing.

In 'cooked' mode, the kernel level terminal driver also does some output translation. And part of that is turning \n into \r\n if needed.

Also, in 'cooked' mode the terminal driver handles special characters like Control-C (sends a SIGINT to the controlling process group (translated by CPython into a KeyboardInterrupt exception)) and Control-Z (sends a SIGTSTP (like a SIGSTOP, but can be caught) to the controlling process group).

In 'cbreak' mode, line editing is no longer done. The terminal driver gives each character (or short character sequence, like the escape sequence for an arrow key) to the program immediately. These characters are not echoed to the screen, and so unless your program then prints them the user won't see them. The terminal driver though still handles special characters like Control-C and Control-Z, though it ceases to handle line editing characters like backspace or the word-erase character (typically Control-W). Also, some output processing is still done, so the driver turns a \n into a \r\n.

In 'raw' mode, no processing is done on either input or output. No special character handling, no echoing, no transforming \n into \r\n, no handling for Control-Z, nothing. It's up to the program that puts the terminal in raw mode to do it all.

Now, you are setting the attributes for sys.stdin so you may think this shouldn't affect sys.stdout. But, in fact, both of your file descriptors lead to the exact same 'instance' of a terminal driver. And it's the settings for the terminal driver that determine what happens. So it doesn't matter if you change those settings through sys.stdin, sys.stdout, or even sys.stderr, all change the same underlying terminal driver instance and they affect all the others.

This, of course, is not true of file descriptors that have been redirected by the shell before your program is launched.

As a side note, you can use the stty -a on the command line to see a full read out of all of these flags (including which control characters result in which signals in cooked and cbreak modes).

Omnifarious
  • 54,333
  • 19
  • 131
  • 194
  • 3
    This is a much better and more detailed answer than mine +1 – TheDavidFactor Jun 28 '18 at 19:48
  • @TheDavidFactor - one thing I missed is that apparently posix has largely done away with the discreet modes at the driver level and instead turned the modes into a finer grain collection of largely independent flags. – Omnifarious Jun 28 '18 at 19:54
10

The Google brought me here when I was searching for an answer to this same question. The clue shared by halex of no carriage return aided my search for the truth. I found my answers in a post on Chris's Wiki: https://utcc.utoronto.ca/~cks/space/blog/unix/CBreakAndRaw which lead me to read the source of tty.py here: https://hg.python.org/cpython/file/618ea5612e83/Lib/tty.py Which brought me to the conclusion that if the goal is to read single characters, instead of:

tty.setraw()

Use:

tty.setcbreak()
TheDavidFactor
  • 1,647
  • 2
  • 19
  • 18
6

Looks like you are only doing a line feed but no carriage return. Change your print to

print("ASD", end="\r\n")
halex
  • 16,253
  • 5
  • 58
  • 67
  • So far this seems to be right; I was under the impression I should never insert `\r` manually under Linux. Thanks – K.Steff Sep 01 '12 at 22:16
  • 1
    @halex Could you explain why exactly this is needed in raw mode but not when printing normally? Thanks. – Stefan Oct 11 '15 at 09:47
  • 1
    The interpretation of some commands differs in raw mode. This is a fossilized convention inherited from terminals and virtual terminals (and, ultimately, teletype machines) of yesteryear. It is retained for backwards compatibility as a de-facto standard. – MRule Sep 05 '21 at 15:16
  • 1
    The answer is completely correct, it woks, and it is not useful in the same time. @MRule pointed out exactly the problem. Thank you. How to keep the print functionality and raw mode simultaneously? ..... – jaromrax Jan 11 '23 at 13:56