1

I'm making a simple shell replacement for Windows called shellpy (for learning purposes). I obviously want the shell to be independent of the Windows CMD, so I'm redirecting all output to StringIO objects. So far it's working pretty well.

But I've run into an issue while running the Python interactive interpreter inside my little shell. Evidently, the interactive interpreter writes everything to stderr except for print output, which it writes to stdout. With my approach, I get output such as this:

shellpy - shellpy $ python
Python 2.7.3 (default, ...) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print "Hello, world!"
H>e>l>l o, world!

>>>

I can't make any sense of this. The way I understand my code is; every time a byte of data is written by the child process' stdout or stderr, it's redirected to the appropriate print function in my shell. How come, then, that the two overlap? Surely the Python interpreter isn't writing to both file descriptors simultaneously? Is my approach flawed? If so, how should I be redirecting the child process' output?

note: I had a very hard time describing my issue in the question title, please feel free to edit it if you can think of something better.

This code launches sub-processes:

child = subprocess.Popen(
    [command] + args,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE
)

# Redirect the child's output to the shell.
redirect_pipe_output(child.stdout, self.print_text)
redirect_pipe_output(child.stderr, self.print_error)

# Wait for the child to finish.
child.wait()

Here is the redirect function, which utilizes a thread to continuously poll the child for output:

def redirect_pipe_output(fileobject, fn):
    """Creates a new thread and continuously tries to read from the input file
    object. When data is read from the file object, `fn` is called with that
    data."""

    def work():
        try:
            while True:
                fn(fileobject.read(1))
        except ValueError:
            pass

    t = threading.Thread(target=work)
    t.daemon = True
    t.start()

The functions self.print_text and self.print_error are currently just wrappers for sys.stdout.write.

Hubro
  • 56,214
  • 69
  • 228
  • 381
  • Are the `>`s and the `hello world` being written to the same place? – Blender Dec 22 '12 at 04:28
  • 1
    @Blender: `>` is written by the interactive Python process to `stderr`; `Hello, world!` is being written by the interactive Python process to `stdout`. – icktoofay Dec 22 '12 at 04:30
  • @icktoofay: Then it looks like Codemonkey's threads are writing to the same place at the same time. – Blender Dec 22 '12 at 04:31
  • @Blender: Even if they were written to the correct places (`stderr` to `stderr`, `stdout` to `stdout`), they would still look like this because (on Unixy systems logged in at a tty) `stdout` and `stderr` are different file descriptors both pointing at the same tty, yielding the same result. I'd assume the same thing would happen on Windows, although I don't know the technical reason why. – icktoofay Dec 22 '12 at 04:33
  • @icktoofay: Reading one byte at a time might be the problem. I remember writing a shell-like script in Python and I could launch and use the Python interpreter from within it. I don't recall havint to read one byte at a time, so maybe replacing `.read(1)` with `.read()` might help. – Blender Dec 22 '12 at 04:40
  • @Blender: `.read()` will keep reading until EOF, meaning that in my case it will *never* return anything. – Hubro Dec 22 '12 at 10:27

0 Answers0