C++ how to flush std:stringbuf?

Question

I need to put the standard output of a process (binary data) to a string buffer and consume it in another thread.

Here is the producer:

while (ReadFile(ffmpeg_OUT_Rd, cbBuffer, sizeof(cbBuffer), &byteRead, NULL)){
    tByte += byteRead;   //total bytes
    sb->sputn(cbBuffer, byteRead);
}
m_bIsFinished = true;
printf("%d bytes are generated.\n", tByte);

Here is the consumer:

while (!PCS_DISPATCHER_INSTANCE->IsFinished())
    Sleep(200);
Sleep(5000);
Mystringbuf* sb = PCS_DISPATCHER_INSTANCE->sb;
printf("Avail: %d\n", sb->in_avail());

It turns out that the consumer cannot get all the bytes of the produced by the producer. ( tByte <> sb->in_avail() )

Is it a kind of internal buffering problem? If yes, how to force the stringbuf to flush its internal buffer?

Answer 1

A streambufhas nothing like flush: writes are done directly into the buffer. There is a pubsync() member that could help, if you would use an object derived such as a filebuf. But this does not apply to your case.

Your issue certainly comes from a a data race on sputn() or is_avail(). Either protect access to the streambuf via a mutex, or via an atomic. If m_bIsFinished is not an atomic, and depending on your implementation of isFinished(), the synchronisation between the threads might not be guaranteed (for example: producer could write to memory, but consumer still obtains an outdated value from the CPU memory cache), which could conduct to such a data race.

Edit:

If you'd experience the issue within a single thread, thus eliminating any potential racing condition, it may come from implementation of streambuf. I could experience this with MSVC13 in a single thread application:

• tracing showed that number of bytes read were accurate, but in_avail() result was always smaller or equal to tByte through the whole loop.
• when reading the streambuf, the correct total number of bytes were read (thus more than indicated by in_avail()).

This behaviour is compliant. According to C++ standard: in_avail() shall return egptr() - gptr() if a read position is available, and otherwhise showmanyc(). The latter is defined as returning an estimate of the number of characters available in the sequence. The only guarantee given is that you could read at least in_avail() bytes without encountering eof.

Workaround use sb->pubseekoff(0,ios::end)- sb->pubseekoff(0,ios::beg); to count the number of bytes available, and make sure you're repositioned at sb->pubseekoff(0,ios::beg) beofre you read your streambuf.