controlling less with Popen

Question

I'm trying to control less from a Python script on Mac OSX. Basically what I would like is to be able to forward control characters (up/down/left/right) but process other input within the Python program. I'm using Popen to start less, but less reads user input from a source other than stdin. Because of that, I'm unsure how to send any characters to less.

The program opens less, waits one second, and then tries to send q to quit using two separate channels: stdin and /dev/tty (since it was mentioned in the SO question I linked above). Neither works.

from subprocess import Popen, PIPE
import time
p1 = Popen("echo hello | less -K -R", stdin=PIPE, shell=True)
time.sleep(1)
p1.stdin.write(bytes('q', 'utf-8'))
with open("/dev/tty", 'w') as tty:
    tty.write('q')
p1.wait()

How can I control less from a Python script?

This smells like an [XY Problem](https://xyproblem.info) - can I ask *why* you want to do this? — match, Jan 20 '18 at 18:26
@match I want to use it to display dictionary entries on the command line, where the workflow should be type word, ENTER -> display entry using `less` -> type word, ENTER -> display with `less`... I don't want to require quitting less with "q" for each lookup. python-pager is too limited, and using less's key file functionality to quit on any key except for arrows would still require the user to press some key to quit before typing in the next word. If there were a way to get the last key pressed within `less` then it would also solve my problem. — Nate Glenn, Jan 20 '18 at 19:01
So you just want to show the first n lines for a second? Would `head` work instead of `less`? — L3viathan, Jan 20 '18 at 20:14
@L3viathan No, the one second pause and quit thing is just for demoing; I didn't want `less` to quit immediately because then it would be difficult to tell if it ran at all. I actually need to control the `less` program, or another suitable pager. — Nate Glenn, Jan 20 '18 at 21:38
Programs that use the tty in raw mode don't work well with pipes. You need to interact with it using a pty and make the parent the master/controlling terminal. — Keith, Jan 20 '18 at 22:42

score 2 · Accepted Answer · answered Jan 20 '18 at 22:25

It's a bit involved, but it's possible to use forkpty(3) to create a new TTY in which you have full control over less, forwarding input and output to the original TTY so that it feels seamless.

The code below uses Python 3 and its standard library. pexpect can do a lot of the heavy lifting but that doesn't ship with Python. Plus it's more educational this way.

import contextlib
import fcntl
import io
import os
import pty
import select
import signal
import struct
import termios
import time
import tty

Assume the rest of the code is indented to run within this context manager.

with contextlib.ExitStack() as stack:

We need to grab the real TTY and set it to raw mode. This can confuse other users of the TTY (for example, the shell after this program exits), so make sure to put it back to the same state after.

tty_fd = os.open('/dev/tty', os.O_RDWR | os.O_CLOEXEC)
stack.callback(os.close, tty_fd)
tc = termios.tcgetattr(tty_fd)
stack.callback(termios.tcsetattr, tty_fd, termios.TCSANOW, tc)
tty.setraw(tty_fd, when=termios.TCSANOW)

Then we can invoke forkpty, which is named pty.fork() in Python. This does a couple things:

Creates a pseudoterminal.
Forks a new child.
Attach the child to the slave end of the PTY.
Return the child's PID and the master end of the PTY to the original process.

The child should run less. Note the use of _exit(2) as it can be unsafe to continue executing other code after a fork.

child_pid, master_fd = pty.fork()
if child_pid == 0:
    os.execv('/bin/sh', ('/bin/sh', '-c', 'echo hello | less -K -R'))
    os._exit(0)
stack.callback(os.close, master_fd)

Then there's a bit of work involved to set up a few asynchronous signal handlers.

SIGCHLD is received when a child process changes state (such as exiting). We can use this to keep track of whether the child is still running.
SIGWINCH is received when the controlling terminal changes size. We forward this size to the PTY (which will automatically send another window change signal to the processes attached to it). We should set the PTY's window size to match to start, too.

It may also make sense to forward signals such as SIGINT, SIGTERM, etc.

child_is_running = True
def handle_chld(signum, frame):
    while True:
        pid, status = os.waitpid(-1, os.P_NOWAIT)
        if not pid:
            break
        if pid == child_pid:
            child_is_running = False
def handle_winch(signum, frame):
    tc = struct.pack('HHHH', 0, 0, 0, 0)
    tc = fcntl.ioctl(tty_fd, termios.TIOCGWINSZ, tc)
    fcntl.ioctl(master_fd, termios.TIOCSWINSZ, tc)
handler = signal.signal(signal.SIGCHLD, handle_chld)
stack.callback(signal.signal, signal.SIGCHLD, handler)
handler = signal.signal(signal.SIGWINCH, handle_winch)
stack.callback(signal.signal, signal.SIGWINCH, handler)
handle_winch(0, None)

Now for the real meat: copying data between the real and fake TTY.

target_time = time.clock_gettime(time.CLOCK_MONOTONIC_RAW) + 1
has_sent_q = False
with contextlib.suppress(OSError):
    while child_is_running:
        now = time.clock_gettime(time.CLOCK_MONOTONIC_RAW)
        if now < target_time:
            timeout = target_time - now
        else:
            timeout = None
            if not has_sent_q:
                os.write(master_fd, b'q')
                has_sent_q = True
        rfds, wfds, xfds = select.select((tty_fd, master_fd), (), (), timeout)
        if tty_fd in rfds:
            data = os.read(tty_fd, io.DEFAULT_BUFFER_SIZE)
            os.write(master_fd, data)
        if master_fd in rfds:
            data = os.read(master_fd, io.DEFAULT_BUFFER_SIZE)
            os.write(tty_fd, data)

It looks straightforward, although I'm glossing over a few things, such as proper short write and SIGTTIN/SIGTTOU handling (partly hidden by suppressing OSError).

Thanks for the writeup! "straightforward", heh. I'll try pexpect and see what I can do. That also contains cross-platform support, which is a plus. — Nate Glenn, Jan 22 '18 at 18:58
@NateGlenn Heh yeah, this is all a bit esoteric. The "straightfoward" part is using a `select` loop, but using `pty`/`termios` is relatively rarer. Some combination of `pexpect.spawn().interact()` should get you most of the way there, you just have to wire up `WINCH` and a few other things you can probably copy from here or the pexpect source code. Good luck! — ephemient, Jan 22 '18 at 20:05

controlling less with Popen

1 Answers1