0

I have a large amount of "input" data that I want to compress to multiple bzip2 streams, without writing data to multiple intermediate files.

Can I put a chunk of input data into a buffer, use BZ2_bzBuffToBuffCompress() to compress that buffer to bzip2-flavored bytes, and repeat with subsequent BZ2_bzBuffToBuffCompress() calls on fresh data, until I have no more input data left?

The bzip2 documentation says the following:

Compression in this manner is a one-shot event, done with a single call to this function. The resulting compressed data is a complete bzip2 format data stream. There is no mechanism for making additional calls to provide extra input data. If you want that kind of mechanism, use the low-level interface.

Does this mean I cannot re-run BZ2_bzBuffToBuffCompress() on newly filled buffer-sized chunks of input data? Should I use BZ2_bzWrite() instead?

EDIT

I actually meant outputting to one archive file, which could contain one or more bzip streams.

Alex Reynolds
  • 95,983
  • 54
  • 240
  • 345
  • My understanding when reading the documentation is that you can not add (append) more data to an already compressed buffer, but creating multiple separate compressed streams or buffers is okay. – Some programmer dude Aug 05 '12 at 13:03

1 Answers1

2

You can keep calling BZ2_bzBuffToBuffCompress() with new data all you like, and you will generate what I think you said you wanted, which was:

data that I want to compress to multiple bzip2 streams

That is exactly what you'll get. Multiple bzip2 streams. Each of those streams then needs to be separately decoded.

If you want to take a series of buffers and make a single bzip2 stream out of them, then you will need to use BZ2_bzCompressInit, BZ2_bzCompress, and BZ2_bzCompressEnd.

Mark Adler
  • 101,978
  • 13
  • 118
  • 158
  • Thanks, I think your second suggestion is what I had in mind. I'll test this out and mark your answer as correct once I've verified this. – Alex Reynolds Aug 05 '12 at 21:30