0

First of all, I'd like to refer to question 29513549, where all respondents seem to agree that it makes sense that the PNG image format was designed to have multiple IDAT chunks in case of larger images - for reading and for writing. These IDAT chunks contain the actual image values.
My question applies to the reading process. In order to reconstruct the values, one needs to first decompress the data with zlib and then to apply inverse filter functions: uncompressed data contain differences to previous values, where possible. So far, I have only come across examples that join all IDATA chunks together - e.g. in this well-written blog of Paul Tan - meaning that all data have to be loaded into memory. I guess that is why the documentation of Python package pypng warns that the read method of the Reader class "may use excessive memory".
I don't know very much about decompression by means of zlib. I know that it's described here, but it seems complicated. This is e.g. because boundaries between IDAT chunks are arbitrary. And it is quite possible for a terminating zlib check value to be split across IDAT chunks. All the same, I'd like to find a way to decompress the IDATA chunks without loading all data into memory at the same time - even if that would maybe mean that each chunk has to be read twice. If there is no way to do this, then the possibility to retrieve multiple IDAT chunks from PNG images is useless. I'm looking forward to your answers.

Dobedani
  • 508
  • 5
  • 20
  • 1
    It's mostly an optimisation for encoders: it doesn't provide any benefit for a well-written decoder, which shouldn't care about the chunk boundaries at all. The `zlib` stdlib module can operate on streams of data. Make a stream, get `zlib` to start decompressing the stream, then pass along the `IDAT` data as you read it from the disk or network. There's no reason to pipe image data to the stream one chunk at a time: just pipe it through as you get it. – smitop Nov 08 '21 at 20:39

1 Answers1

2

zlib streams, so you can simply feed an IDAT chunk at a time to zlib's inflator and get the decompressed data so far back out.

Mark Adler
  • 101,978
  • 13
  • 118
  • 158