NSTask requires flush when reading from a process' stdout, Terminal does not

Question

I have a simple Python script that asks for your name, then spits it back out:

def main():
    print('Enter your name: ')
    for line in sys.stdin:
        print 'You entered: ' + line

Pretty simple stuff! When running this in the OS X Terminal, it works great:

$ python nameTest.py 
Enter your name: 
Craig^D
You entered: Craig

But, when attempting to run this process via an NSTask, the stdout only appears if additional flush() calls are added to the Python script.

This is how I have my NSTask and piping configured:

NSTask *_currentTask = [[NSTask alloc] init];
_currentTask.launchPath = @"/usr/bin/python";
_currentTask.arguments = [NSArray arrayWithObject:@"nameTest.py"];

NSPipe *pipe = [[NSPipe alloc] init];
_currentTask.standardOutput = pipe;
_currentTask.standardError = pipe;

dispatch_queue_t stdout_queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0);

__block dispatch_block_t checkBlock;

checkBlock = ^{
    NSData *readData = [[pipe fileHandleForReading] availableData];
    NSString *consoleOutput = [[NSString alloc] initWithData:readData encoding:NSUTF8StringEncoding];
    dispatch_sync(dispatch_get_main_queue(), ^{
        [self.consoleView appendString:consoleOutput];
    });
    if ([_currentTask isRunning]) {
        [NSThread sleepForTimeInterval:0.1];
        checkBlock();
    } else {
        dispatch_sync(dispatch_get_main_queue(), ^{
            NSData *readData = [[pipe fileHandleForReading] readDataToEndOfFile];
            NSString *consoleOutput = [[NSString alloc] initWithData:readData encoding:NSUTF8StringEncoding];
            [self.consoleView appendString:consoleOutput];
        });
    }
};

dispatch_async(stdout_queue, checkBlock);

[_currentTask launch];

But when running the NSTask, this is how it appears (it is initially blank, but after entering my name and pressing CTRL+D, it finishes all at once):

Craig^DEnter your name: 
You entered: Craig

So, my question is: How can I read the stdout from my NSTask without requiring the additional flush() statements in my Python script? Why does the Enter your name: prompt not appear immediately when run as an NSTask?

Answer 1

When Python sees that its standard output is a terminal, it arranges to automatically flush sys.stdout when the script reads from sys.stdin. When you run the script using NSTask, the script's standard output is a pipe, not a terminal.

UPDATE

There is a Python-specific solution to this. You can pass the -u flag to the Python interpreter (e.g. _currentTask.arguments = @[ @"-u", @"nameTest.py"];), which tells Python not to buffer standard input, standard output, or standard error at all. You can also set PYTHONUNBUFFERED=1 in the process's environment to achieve the same effect.

ORIGINAL

A more general solution that applies to any program uses what's called a “pseudo-terminal” (or, historically, a “pseudo-teletype”), which we shorten to just “pty”. (In fact, this is what the Terminal app itself does. It is a rare Mac that has a physical terminal or teletype connected to a serial port!)

Each pty is actually a pair of virtual devices: a slave device and a master device. The bytes you write to the master, you can read from the slave, and vice versa. So these devices are more like sockets (which are bidirectional) than like pipes (which are one-directional). In addition, a pty also let you set terminal I/O flags (or “termios”) that control whether the slave echoes its input, whether it passes on its input a line at a time or a character at a time, and more.

Anyway, you can open a master/slave pair easily with the openpty function. Here's a little category that you can use to make an NSTask object use the slave side for the task's standard input and output.

NSTask+PTY.h

@interface NSTask (PTY)

- (NSFileHandle *)masterSideOfPTYOrError:(NSError **)error;

@end

NSTask+PTY.m

#import "NSTask+PTY.h"
#import <util.h>

@implementation NSTask (PTY)

- (NSFileHandle *)masterSideOfPTYOrError:(NSError *__autoreleasing *)error {
    int fdMaster, fdSlave;
    int rc = openpty(&fdMaster, &fdSlave, NULL, NULL, NULL);
    if (rc != 0) {
        if (error) {
            *error = [NSError errorWithDomain:NSPOSIXErrorDomain code:errno userInfo:nil];
        }
        return NULL;
    }
    fcntl(fdMaster, F_SETFD, FD_CLOEXEC);
    fcntl(fdSlave, F_SETFD, FD_CLOEXEC);
    NSFileHandle *masterHandle = [[NSFileHandle alloc] initWithFileDescriptor:fdMaster closeOnDealloc:YES];
    NSFileHandle *slaveHandle = [[NSFileHandle alloc] initWithFileDescriptor:fdSlave closeOnDealloc:YES];
    self.standardInput = slaveHandle;
    self.standardOutput = slaveHandle;
    return masterHandle;
}

@end

You can use it like this:

NSTask *_currentTask = [[NSTask alloc] init];
_currentTask.launchPath = @"/usr/bin/python";
_currentTask.arguments = @[[[NSBundle mainBundle] pathForResource:@"nameTest" ofType:@"py"]];

NSError *error;
NSFileHandle *masterHandle = [_currentTask masterSideOfPTYOrError:&error];
if (!masterHandle) {
    NSLog(@"error: could not set up PTY for task: %@", error);
    return;
}

Then you can read from the task and write to the task using masterHandle.