What can you do with a pty?

Question

Having read various resources including http://www.linusakesson.net/programming/tty/ I am still confused and curious about the structure and use of pseudo-terminals In a linux terminal (bash not tty) we have three streams:

• stdin
• stdout
• stderr

There is a file descriptor for each one. These file descriptors map to streams FILE*. For example, we can use fileno() and fdopen() to switch between them.

What can I do with a pty?

I normally think of stdout, stdin and stderr being associated with a process. I don't think it is correct to say that there is a stdout, stdin and stderr associated with a pty. So given a pty (for a process like bash rather than its pid) how could you obtain them and wire them up to a terminal in the bash sense?

How does a single file descriptor (for the pty) map to three streams? (clearly it doesn't) Can it map to a process somehow?

When using openpty(), I think to connect the processes on the master and slave terminals you need to set up separate pipes for stdin, stdout and stderr and have a select loop connect them and transfer data between them.

But could you instead assume that stdin and stdout are connected (via the tty) and just set up a stderr pipe to avoid stdout and stderr being merged?

If we have a master and slave process what does the 'extra' tty actually do for us?

---

Here's some things I think I know and are relevant:

The openpty() function returns a pair of file descriptors representing the master and slave ends of a pseudo terminal.

Each file descriptor points to character device - see https://man7.org/linux/man-pages/man7/pty.7.html

Now devices in Unix are just special files so a file descriptor makes sense.

A bash terminal has three streams but a tty is just a single channel (no distinction between stdout and stderr). A virtual terminal (tty) also has some 'magic' (for example, a line discipline).

If I was to ask what file does the FD for a pty map to it would be the device file /dev/ptyX or whatever.

There is a 'terminal' at either end of the master/slave channel created by a openpty().

What data does it actually transfer?

I normally think of a file descriptor as an index into a table of kernel structures representing files. Clearly some of those files are streams (an kernel version of FILE*) and some of them are special.

The main userspace structure associated with a terminal (tty) is termios obtained from an FD using tcgetattr. This lets you set parameters for the terminal but it does not identify any streams or processes or the line discipline.

With a FILE* or an fd you think is a file there are a very clear and well trodden set of APIs defining what you can do.

I vaguely know what I can do with a character device (e.g. https://unix.stackexchange.com/questions/37829/how-do-character-device-or-character-special-files-work).

But what can you do with a pty?

Given a stream we can determine if it has an associated tty using isatty() and identify the tty using ttyname(). So internally somewhere there is something that knows that a tty and stream are associated. Also isatty(STDERR_FILENO) is true, so its not just stdin and stdout.

---

What originally piqued my interest is a program which should be a daemon but creates an ncurses UI. I have been thinking about how you could make the process open its ncurses on demand UI (in a pty). The screen program lets you do this but what does it do under the hood.

A question describing this use case is - attach a terminal to a process running as a daemon (to run an ncurses UI)

---

Another way to look it this question is from object oriented perspective.

What functions are there that take a 'pty' as an argument. What are they used for? If I had a pty object. What methods would it have?

Some crucial pieces of information are missing from the man pages if you just list these functions. What are the pre-conditions, post-conditions & invariants.

Answer 1

File descriptors are per-process identifiers, that refer to kernel-internal file descriptions. File description includes position for regular files, and whether it was opened for read, write, both, or for only path operations. These are called file status flags in the fcntl() man page. Each file descriptor has its own descriptor flags (currently only one, close-on-exec, O_CLOEXEC/FD_CLOEXEC).

The FILE streams are a standard C library abstraction. If we ignore the GNU fopencookie() extension, all C streams are associated with exactly one file descriptor, and only one C stream can be associated with a given file descriptor. (Of course, you can duplicate a file descriptor, and use fdopen() to associate a FILE stream with the new duplicate file descriptor. However, the C library does not know that the two separate streams now share the underlying file position, which can cause problems when the descriptor refers to a file. When the descriptor refers to a device, say a terminal or pseudoterminal, and the original descriptor was opened in a compatible mode (read/write/both), there are no problems.)

A pseudoterminal is basically a bidirectional pipe, with the kernel termios layer in between. A typical process running in a pseudoterminal has all three standard descriptors referring to the same file description: the slave end of the pseudoterminal.

The termios layer does not just do minor processing on the data read and written by the slave end: it can also cause signals to raised. Each pseudoterminal is also associated with a size (number of rows and columns, in characters). When the master end of a pseudoterminal changes the size, each process in the foreground process group (explained below) will receive a SIGWINCH signal. If the master closes the pseudoterminal, all processes having that as their controlling terminal will receive a SIGHUP signal. (This is also why/how closing a terminal window while you have a command or program running in it, will usually kill that command or program.)

Each process can have at most one controlling terminal. Each terminal can be associated with at most one foreground process group (managed via tcgetpgrp() and tcsetpgrp()). The concepts of session (getsid(), setsid(), foreground process group, and background process group, all involve a terminal (or a pseudoterminal).

When a process has all three standard C FILE streams connected to a terminal or pseudoterminal, standard input receives data from the master (via the termios layer), and both standard output and standard error send data to the master (via the termios layer). So, standard input, output, and error are not connected to each other directly, only to the same terminal/pseudoterminal slave end.

---

Let's say you are a GTK+3 C programmer, and you want to create your own terminal window application, like xterm or gnome-terminal.

The application would create a pseudoterminal for each window. Whenever it receives keyboard events, it would write the corresponding key sequence (1 for 1, : for :, and so on) to the master pseudoterminal descriptor. Everything it would read from the master pseudoterminal descriptor, it would render in the window. For the slave side, the application forks a child process, opens the slave side for all three standard streams, and either executes the equivalent of /bin/login -f $(id -un) with root privileges, or does the equivalent for the current user itself. Essentially, certain housekeeping is done for the user, and finally a login shell is opened for the user in their home directory.

Most terminal emulators support ANSI escape codes and xterm extensions to those. Terminal info database and ncurses applications rely on the TERM environment variable to be set to the current terminal type. (xterm-256color is probably the most common in Linux, but others are also supported. The terminal emulator is expected to set the correct TERM environment variable when starting processes on the slave side; there is no automagic detection per se.)

All things you need a pseudoterminal for, involve a terminal-like environment. It could be facing a human user (as in the above GUI terminal application), or it could be facing an application like nano, vim, emacs, links, or lynx that works in a terminal, and acting on behalf of an user.

The way screen works is that it maintains the pseudoterminal the applications are running in even when the user is not connected to that screen process. When the user wants to detach (i.e., disconnect but let the processes keep running), the screen application (the master side of the pseudoterminal it runs the processes under) detaches from the terminal/pseudoterminal the human is using, but keeps running. To resume, the screen -r command (and keyboard shortcuts when already running screen) reconnects the user terminal or pseudoterminal to the screen application.

Essentially, with such terminal multiplexers you have the real user terminal or pseudoterminal (which could be a terminal emulator like xterm or gnome-terminal, or even a non-terminal process like a remote SSH connection to this machine), connected to the multiplexer application, which relays keypresses and outputs when attached to a pseudoterminal (whose master the multiplexer application is), with the shell or other applications running under that pseudoterminal.