5

Is there some magic required to obtain a "zlib sync flush" when using boost::iostreams::zlib_compressor ? Just invoking flush on the filter, or strict_sync on a filtering_ostream containing it doesn't see to do the job (ie I want the compressor to flush enough that the decompressor can recover all the bytes consumed by the compressor so far, without closing the stream).

Looking at the header, there seem to be some "flush codes" defined (notably a sync_flush) but it's unclear to me how they should be used (bearing in mind my compressor is just added into a filtering_ostream).

timday
  • 24,582
  • 12
  • 83
  • 135

2 Answers2

2

It turns out there is a fundamental problem that the symmetric_filter that zlib_compressor inherits from isn't itself flushable (which seems rather an oversight).

Possibly adding such support to symmetric_filter would be as simple as adding the flushable_tag and exposing the existing private flush methods, but for now I can live with it.

timday
  • 24,582
  • 12
  • 83
  • 135
1

This C++ zlib wrapper library, of which I'm the author, supports flush functionality and is arguably simpler to use:

https://github.com/rudi-cilibrasi/zlibcomplete

It is as easy as this:

#include <iostream>
#include <zlc/zlibcomplete.hpp>

using namespace zlibcomplete;
using namespace std;

int main(int argc, char **argv)
{
  const int CHUNK = 16384;
  char inbuf[CHUNK];
  int readBytes;
  ZLibCompressor compressor(9, auto_flush);
  for (;;) {
    cin.read(inbuf, CHUNK);
    readBytes = cin.gcount();
    if (readBytes == 0) {
      break;
    }
    string input(inbuf, readBytes);
    cout << compressor.compress(input);
  }
  cout << compressor.finish();
  return 0;
}

The main difference from boost is that instead of using a template class filter you simply pass in a string and write out the compressed string that results as many times as you want. Each string will be flushed (in auto_flush mode) so it can be used in interactive network protocols. At the end just call finish to get the last bit of compressed data and a termination block. While the boost example is shorter, it requires using two other template classes that are not as well-known as std::string, namely filtering_streambuf and the less-standard boost::iostreams:copy. The boost interface to zlib is incomplete in that it does not support Z_SYNC_FLUSH. This means it is not appropriate for online streaming applications such as in TCP interactive protocols. I love boost and use it as my main C++ support library in all of my C++ projects but in this particular case it was not usable in my application due to the missing flush functionality.

Rudi Cilibrasi
  • 885
  • 4
  • 12