pipe, fork, and non-blocking IPC

Question

So I was trying to run a system command (or exec, or whatever) at a child process after a fork(), and push some input to it, and then get its output. It looks like this after the fork(), and pc and cp is parent-child and child-parent pipes.

  case 0:
        /* Child. */
        close(STDOUT_FILENO); /* Close current stdout. */
        dup2(cp[1], STDOUT_FILENO);

        close(STDIN_FILENO);
        dup2(pc[0], STDIN_FILENO);

        close( pc[1]);
        close( cp[0]);
        execlp("cat", "cat", NULL);
        exit(1);
  default:
        /* Parent. */
        /* Close what we don't need. */
        printf("Input to child:\n");

        string theinput("Hey there baby");
        write(pc[1], theinput.c_str(), theinput.size());
        close(pc[1]);

        cout << "The input : " << theinput << endl;


        printf("\nOutput from child:\n");
        close(cp[1]);
        while( read(cp[0], &ch, 1) == 1)
        {
           write(1, &ch, 1);
           outcount++;
        }

        exit(0);

Now, it seems to work nicely (if you want the code : http://pastebin.com/Fh7GrxYm ), but when I was talking at #posix on irc, they were going mad, about how this can potentially block, and how it "depends on how the kernel is feeling".

There was an msdn blog post about the same thing: http://blogs.msdn.com/b/oldnewthing/archive/2011/07/07/10183884.aspx

How can one prevent blocking, if any, etc?

score 1 · Answer 1 · answered Dec 29 '11 at 09:03

In your case, the deadlock may occur when the parent process reaches this line:

write(pc[1], theinput.c_str(), theinput.size());

If "theinput" is a lot of data then the parent process may fill up the pc pipe. The child process (cat here) might read some of it but not all of it. cat will them echo it back to you. But again if it's a lot of data, it may fill up the cp pipe and will block until someone reads the data out of that pipe. Which will never happen because the parent process is blocked waiting the the pc pipe to drain and will never reach the code that consumes the contents of the cp pipe. DEADLOCK.

Like your IRC buddies said, whether this happens or not depends on a lot of things such as the amount of data involved, the amount of data a pipe can accommodate before it blocks (a kernel-dependent parameter), the amount of stdio or other buffering performed by the parent or child process, etc...

Your options are:

Use two processes to control the external command: one that feeds it data and one that reads the results back. You'll have to fork() twice for this. This looks a lot like a shell pipeline. Conventionally, the ultimate data source is the grandchild process, the filter is the middle parent, and the ultimate data sink of the grantparent process.
Use two threads to control the external command. Similar to the previous option.
Use non-blocking I/O to control the external command. Set both file descriptors to non-blocking mode using fcntl() and set up an event loop using poll() or select() to wait for either file descriptor to be ready. When either file descriptor is ready, be prepared for write() to complete only partially and for read() to not read everything at once.
Use an existing event loop like glib's, set up your pipes as IO Channels, and watch them to know when it's time to read or write data. Similar to the previous option but uses an existing framework so you can integrate with an existing application event loop.

BTW: Your exit(1) should be _exit(1) to prevent the C library from inappropriately invoking exit-time hooks in the short-lived child process.

pipe, fork, and non-blocking IPC

1 Answers1