1

I need to add more things onto a stacklike structure, but compress them additively such that addition of new data results in the expected compression gain, but new chunks are still stored as compressed data without altering any of the past data chunks.

So in other words, it must preserve the additive property over successive compressed chunks. If f() is the 'adding another chunk' function, I need the following to hold for all chunks 'x':

\sum_{i=1}^{n}f(x_i)=f(\sum_{i=1}^{n}x_i)

starball
  • 20,030
  • 7
  • 43
  • 238
Andy
  • 67
  • 1
  • 7

1 Answers1

1

Sure. Deflate does this, if you just keep it running and terminate each chunk at a byte boundary, e.g. with Z_SYNC_FLUSH, so that it can be written to file up to and including that chunk.

So long as your chunks are large enough, you will get the same compression gain.

Mark Adler
  • 101,978
  • 13
  • 118
  • 158
  • Excellent, thanks! I tried to verify something like this by looking at successive ZIP files but it's probably going to be better to look at the Deflate algorithm at a lower level. Cheers – Andy Jan 17 '23 at 01:45