12

I am trying to find some information on data limits related to stdout on Windows. I can't seem to find the information on MSDN.

  1. Is there a limit to how much data can be written to stdout? If so, what happens if the limit is reached? Is the data lost?

  2. If stdout is redirected (for example, by launching the process from .Net and using the ProcessStartInfo.RedirectStandardOutput property), does that have any effect on how much data can be written? As I read from the stdout stream in the calling process, does that affect the limitations?

  3. Are these limits related in any way to named pipes?

jameswelle
  • 1,367
  • 1
  • 15
  • 26

3 Answers3

19

It depends where it's going - but yes, if you redirect the output in .NET you can easily run into problems if you don't read the output. When the buffer runs out, writes to stdout in the child process will block. One common-ish cause of deadlock is a "parent" process waiting for the "child" to exit, and then reading the output - that won't work if the child needs the parent to read the output to free up buffer space.

.NET has made this slightly easier by allowing an event-driven approach with Process.OutputDataReceived and Process.ErrorDataReceived. This means you don't need to start two threads (one to read stdout, one to read stderr) just to keep the process from blocking...

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • 1
    That's WAY more helpful than what I posted. Deleting my own post ad voting you up. – David Jan 14 '10 at 17:55
  • Now that's a handy feature. I wonder if Java will add something like that someday. – Michael Myers Jan 14 '10 at 17:57
  • (+1) Its probably far more useful to have the .NET perspective as I'm sure the decisions made are to simplify things for the programmer -- allthough stdout is quite simple (by default) in C as well. – Hassan Syed Jan 14 '10 at 18:02
  • Cool, thanks! Is it the same in a Win32 environment? For example, if I run "process1.exe | process2.exe" from the command line, process1 will be able to write to stdout until the buffer is full and then will block until process2 has read enough data so that it can write again? I haven't seen any information on this on MSDN. Is it available anywhere with the buffer size, etc? – jameswelle Jan 14 '10 at 18:35
  • How do we use this in PowerShell, when a child process is started with System.Diagnostics.Process, and stderr and stdout redirected? – Adarsha May 06 '16 at 03:06
  • @Adarsha: Redirected in what terms? Have you tried just using those members, calling `BeginOutputReadLine` and BeginErrorReadLine` to kick things off? I don't know what difference PowerShell would make. If this doesn't work, I suggest you ask a new question with a [mcve]. – Jon Skeet May 06 '16 at 05:14
4

Some things to keep in mind:

1) Jon is right - if the buffer limit is reached, the write call in your subprocess will block. You need to drain the stdout stream if it is not being redirected somewhere that will cause it to automatically drain - like a file. Pipes need to be drained, and usually, if you can "attach" to a subprocess' output, you're attaching to a pipe.

2) The I/O to an output stream is probably buffered, which means that if the subprocess writes some information to stdout without explicitly calling flush(), which is almost always the case, you might not see the output. Flush is automatically called when the process exits, so if it's a short small subprocess you should be OK, but if it's not, you have no real way of forcing its output to show up when you want it to.

3) Named pipes are essentially a buffer that the OS maintains that can be written to and read from - that is, they're like a file you can write to from one process and read from in another, without actually having the overhead of having a file on disk. Very handy for communication between processes, but all the I/O limitations with buffering / full buffers still apply.

Matt
  • 10,434
  • 1
  • 36
  • 45
3

stdout has a buffer of 1024 bytes

FSD
  • 89
  • 4