0

I'm using I/O completion ports for a process management library (yes, there's a reason for this). You can find the source for what I'm talking about here: https://github.com/jcommon/process/blob/master/src/main/java/jcommon/process/platform/win32/Win32ProcessLauncher.java (take a look at lines 559 and 1137 -- yes, that class needs to be refactored and cleaned up).

I'm launching a child process and using named pipes (not anonymous pipes b/c I need asynchronous, overlapped ReadFile()/WriteFile()) in order to process the child process' stdout and stderr. This is mostly actually working. In a test, I launch 1,000 concurrent processes and monitor their output, ensuring they emit the proper information. Typically either all 1,000 work fine or 998 of them, leaving a couple which have problems.

Those couple of processes are showing that not all their messages are being received. I know the message is being output, but the thread processing GetQueuedCompletionStatus() for that process returns from the read with ERROR_BROKEN_PIPE.

The expected behavior is that the OS (or the C libs) would flush any remaining bytes on the stdout buffer upon process exit. I would then expect for those bytes to be queued to my iocp before getting a broken pipe error. Instead, those bytes seem to disappear and the read completes with an ERROR_BROKEN_PIPE -- which in my code causes it to initiate the teardown for the child process.

I wrote a simple application to test and figure out the behavior (https://github.com/jcommon/process/blob/master/src/test/c/stdout-1.c). This application disables buffering on stdout so all writes should effectively be flushed immediately. Using that program in my tests yields the same issues as launching "cmd.exe /c echo hi". And at any rate, shouldn't the application (or OS?) flush any remaining bytes on stdout when the process exits?

The source is in Java, using direct-mapped JNA, but should be fairly easy for C/C++ engineers to follow.

Thanks for any help you can provide!

David Hoyt
  • 11
  • 2

1 Answers1

0

Are you sure that the broken pipe error isn't occurring with a non zero ioSize? If ioSize is not zero then you should process the data that was read as well as noting that the file is now closed.

My C++ code which does this basically ignores ERROR_BROKEN_PIPE and ERROR_HANDLE_EOF and simply waits for either the next read attempt to fail with one of the above errors or the current read to complete with zero bytes read. The code in question works with files and pipes and I've never seen the problem that you describe when running the kind of tests that you describe.

Len Holgate
  • 21,282
  • 4
  • 45
  • 92