Named Pipes, How to know the exact number of bytes to read on Reading side. C++, Windows

Question

I am using Named Pipes configured to send data as a single byte stream to send serialized data structures between two applications. The serialized data changes in size quite dramatically. On the sending side, this is not a problem, I can adjust the number of bytes to send exactly.

How can I set the buffer on the receiveing (Reading) end to the exact number of bytes to read? Is there a way to know how big the data is on the sending (Writing) side is?

I have looked at PeekNamedPipe, but the function appears useless for byte typed named pipes?

lpBytesLeftThisMessage [out, optional] A pointer to a variable that receives the number of bytes remaining in this message. This parameter will be zero for byte-type named pipes or for anonymous pipes. This parameter can be NULL if no data is to be read.

http://msdn.microsoft.com/en-us/library/windows/desktop/aa365779(v=vs.85).aspx

How does one handle such a situation best if you cannot determine the exact required buffer size?

Sending Code

    string strData;
strData =  "ShortLittleString";

DWORD numBytesWritten = 0;
result = WriteFile(
    pipe, // handle to our outbound pipe
    strData.c_str(), // data to send
    strData.length(), // length of data to send (bytes)
    &numBytesWritten, // will store actual amount of data sent
    NULL // not using overlapped IO
);

Reading Code:

DWORD numBytesToRead0 = 0;
DWORD numBytesToRead1 = 0;
DWORD numBytesToRead2 = 0;

BOOL result = PeekNamedPipe(
pipe,
NULL,
42,
&numBytesToRead0,
&numBytesToRead1,
&numBytesToRead2
);

char * buffer ;

buffer = new char[numBytesToRead2];

char data[1024]; //1024 is way too big and numBytesToRead2 is always 0 
DWORD _numBytesRead = 0;

BOOL    result = ReadFile(
pipe,
data, // the data from the pipe will be put here
1024, // number of bytes allocated
&_numBytesRead, // this will store number of bytes actually read
NULL // not using overlapped IO
);

In the code above buffer is always of size 0 as the PeakNamedPipe function returns 0 for all numBytesToRead variables. Is there a way to set this buffer size exactly? If not, what is the best way to handle such a situation? Thanks for any help!

Answer 1

Why do you think you could not use lpTotalBytesAvail to get sent data size? It always works for me in bytes mode. If it's always zero possibly you did something wrong. Also suggest to use std::vector as data buffer, it's quite more safe than messing with raw pointers and new statement.

lpTotalBytesAvail [out, optional] A pointer to a variable that receives the total number of bytes available to be read from the pipe. This parameter can be NULL if no data is to be read.

Sample code:

// Get data size available from pipe
DWORD bytesAvail = 0;
BOOL isOK = PeekNamedPipe(hPipe, NULL, 0, NULL, &bytesAvail, NULL);
if(!isOK)
{
   // Check GetLastError() code
}

// Allocate buffer and peek data from pipe
DWORD bytesRead = 0;    
std::vector<char> buffer(bytesAvail);
isOK = PeekNamedPipe(hPipe, &buffer[0], bytesAvail, &bytesRead, NULL, NULL);
if(!isOK)
{
   // Check GetLastError() code
}

Answer 2

Well, you are using ReadFile(). The documentation says, among other things:

If a named pipe is being read in message mode and the next message is longer than the nNumberOfBytesToRead parameter specifies, ReadFile returns FALSE and GetLastError returns ERROR_MORE_DATA. The remainder of the message can be read by a subsequent call to the ReadFile or PeekNamedPipefunction.

Did you try that? I've never used a pipe like this :-), only used them to get to the stdin/out handles of a child process.

I'm assuming that the above can be repeated as often as necessary, making the "remainder of the message" a somewhat inaccurate description: I think if the "remainder" doesn't fit into your buffer you'll just get another ERROR_MORE_DATA so you know to get the remainder of the remainder.

Or, if I'm completely misunderstanding you and you're not actually using this "message mode" thing: maybe you are just reading things the wrong way. You could just use a fixed size buffer to read data into and append it to your final block, until you've reached the end of the data. Or optimize this a bit by increasing the size of the "fixed" size buffer as you go along.