A palette base PNG with IDAT that has BTYPE=00 for no compression, now with Adler32 code

Question

I am working on some code that builds simple palette based PNG files without libpng. The output file, at this stage only has IHDR, PLTE, IDAT(x3) and IEND chunks. The only thing which is possibly a bit different in that the pixel index values in the IDAT chunk are not compressed, that is the various zlib / block header bytes are as follows.

CMF = 0x78.
FLG = 0x9C (have also a few other values here but always with bit 5 clear).
Block header byte = 0x01 (BFINAL = 1, BTYPE = 00).

From what I can see the code builds the file correctly, however some image viewers refuse to display the image completely, if at all.

MS Paint is happy.
GIMP is happy.
LibreOffice Draw is happy.
Ristretto >> Fatal error reading PNG image file: Not enough compressed data.
ImageMagick >> identify: Not enough image data `20160317_PNG_064.png' @ error/png.c/MagickPNGErrorHandler/1645.
Eye of Gnome >> not enough image data.

I have put the file through a few different tools, again with mixed results.

optipng >> not enough image data.
pngchunks does not report any errors.
pngcheck does not report any errors.

Here is the hex view of the file 20160317_PNG_064.png

The picture it does generate is this small 8x8 pixel image.

So I am at a bit of a dead end as to what to try next. Any and all assistance is appreciated.

EDIT_000 Having narrowed down the issue to the Adler32 calculation here is, as requested by @Mark Adler, the code that I am using to calculate the Adler32 value with test data in the main function. Btw, it is not fancy and I code very verbose.

#include <stdio.h>

#define     DEBUG

static const unsigned long GC_ADLER32_BASE = 0xFFF1;   // Largest prime smaller than 65536 is 65521.

unsigned long Adler32_Update
        (
        unsigned long Adler32,
        unsigned char *Buffer,
        unsigned int BufferLength
        )
{
    unsigned long   ulW0;
    unsigned long   ulW1;
    unsigned int    uiW0;
#ifdef DEBUG
    printf("\n");
    printf("        Incoming Adler32 value.................0x%.8X\n", Adler32);
#endif
    ulW0 = Adler32 & 0xFFFF;
    ulW1 = (Adler32 >> 0x0010) & 0xFFFF;
#ifdef DEBUG
    printf("        Inital sum values are..................0x%.8X, 0x%.8X\n", ulW0, ulW1);
#endif
    for (uiW0 = 0x0000; uiW0 < BufferLength; uiW0 = uiW0 + 0x0001)
        {
        ulW0 = (ulW0 + Buffer[uiW0]) % GC_ADLER32_BASE;
        ulW1 = (ulW1 + ulW0) % GC_ADLER32_BASE;
        }
#ifdef DEBUG
    printf("        Final sum values are...................0x%.8X, 0x%.8X\n", ulW0, ulW1);
#endif
    Adler32 = (ulW1 << 0x0010) | ulW0;
#ifdef DEBUG
    printf("        Outgoing Adler32 value.................0x%.8X\n", Adler32);
#endif
    return (Adler32);
}


unsigned long Adler32_Get
        (
        unsigned char *Buffer,
        unsigned int BufferLength
        )
{
    unsigned long   Adler32;

    Adler32 = 0x00000001L;
    Adler32 = Adler32_Update(Adler32, Buffer, BufferLength);
    return (Adler32);
}


int main
    (
    unsigned int    argc,
    unsigned char   *arg[]
    )
{
    unsigned long   Adler32;
    unsigned char data[272] = 
        {
    0x00, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
    0x01, 0x00, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 
    0x01, 0x01, 0x00, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x01, 0x01, 0x01, 0x01, 0x01, 
    0x01, 0x01, 0x01, 0x00, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x01, 0x01, 0x01, 0x01, 
    0x01, 0x01, 0x01, 0x01, 0x00, 0x02, 0x02, 0x02, 0x02, 0x01, 0x01, 0x01, 0x01, 0x00, 0x00, 0x00, 
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x02, 0x02, 0x02, 0x01, 0x01, 0x01, 0x01, 0x00, 0x00, 
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x02, 0x02, 0x02, 0x01, 0x01, 0x01, 0x01, 0x00, 
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x02, 0x02, 0x02, 0x01, 0x01, 0x01, 0x01, 
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01, 0x01, 0x01, 0x02, 0x02, 0x02, 
    0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x00, 0x01, 0x01, 0x01, 0x01, 0x02, 0x02, 
    0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x00, 0x01, 0x01, 0x01, 0x01, 0x02, 
    0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x00, 0x01, 0x01, 0x01, 0x01, 
    0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x00, 0x01, 0x01, 0x01,
    0x01, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01, 0x01, 0x01, 0x02, 0x02, 0x02, 0x02, 0x00, 0x01, 0x01, 
    0x01, 0x01, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01, 0x01, 0x01, 0x02, 0x02, 0x02, 0x02, 0x00, 0x01, 
    0x01, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01, 0x01, 0x01, 0x02, 0x02, 0x02, 0x02, 0x00, 
    0x01, 0x01, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01, 0x01, 0x01, 0x02, 0x02, 0x02, 0x02
        };
    Adler32 = Adler32_Get(data, sizeof(data));
    printf("\n");
    printf("The Adler32 value is ..........................0x%.8X\n", Adler32);
    return(0x00);
}

Are you sure your chunk of `data` represents the uncompressed data in that same image? An 8x8 palette image should be, if I calculate it correctly, 8x8 = 64 bytes long. Every row has a filter byte before it so that adds up to a mere 72 bytes. If I divide your data up in lines starting with a `0` filter byte, I get a 16 x 16 image (which, plus 16 row filters, adds up to the 272 shown here). — Jongware, Mar 20 '16 at 00:52
(After testing) Yup. If I feed the original 72 bytes into your current Adler32 routine, I get `Outgoing Adler32 value.................0x10080061` – *precisely* what dr. Adler calculated. So your routine seems correct after all, you were just applying it on a wrong set of data. — Jongware, Mar 20 '16 at 01:06

Mark Adler · Accepted Answer · 2016-03-18T05:29:33.877

3

Whatever is calculating the Adler-32 value is incorrect. The correct Adler-32 value for the data is 0x10080061, which should be stored in the stream as 10 08 00 61. If I fix that in the linked file, and make a new CRC for that chunk, then all is good.

The fixed image is:

edited Mar 18 '16 at 05:29

answered Mar 18 '16 at 05:20

Mark Adler

101,978
13
118
158

Once again I have to offer my thanks, especially to Dr @Mark Adler. In my source, forcing these values fixes the issues. I just now have to find where the wheel are coming off the billy cart. At the present the Adler-32 gets the block header and all the scanline filter bytes. I suspect that either one or both of these are not suppose to be passed to the Adler-32 calculation. – SkyPirate Mar 18 '16 at 06:16
It would appear that the filter bytes are included in the data stream that is presented to the Adler32 calculation; the block header byte and LEN and NLEN bytes are not. With these changes all works good for the 8x8 pixel image. However if I then make the image any larger, things run aground once more. Using MS Excel to calculate the Adler 32 value, it matches that in the code so there is still something not quite right. Happy to supply a 16x16 pixel hex file if anyone would like to provide further assistance. – SkyPirate Mar 18 '16 at 10:44
1

What you pass to the Adler-32 is simply the uncompressed data in the zlib stream. Perhaps you can post your Adler-32 code. – Mark Adler Mar 18 '16 at 15:04

score 2 · Answer 2 · answered Mar 17 '16 at 20:40

2

As a co-author of the PNG spec, I believe this file is entirely compliant. The PNG spec defers definition of "deflate" to RFC 1951, and that RFC explicitly allows BTYPE=0 sections. You may want to contact the authors of those programs that have trouble with it and report a bug, attaching your file as sample input.

answered Mar 17 '16 at 20:40

Lee Daniel Crocker

12,927
3
29
55

1

The error message that I'm getting from ImageMagick and from pngcrush is "incorrect data check" which is talking about the computed adler32 checksum not matching the last 4 bytes of the zlib datastream. Note that "pngcheck" doesn't check for this so wouldn't report an error. Firefox displays the image while issuing a warning "W/PNGDecoder libpng warning: Truncated compressed data in IDAT" – Glenn Randers-Pehrson Mar 17 '16 at 21:00
Ah, I didn't check that either. – Lee Daniel Crocker Mar 17 '16 at 21:11
@ Glenn Randers-Pehrson - So that I am not shooting in the breeze, are you able to provide the calculated value is for comparison? I have a number of different Adler32 implementations from different sources, and unfortunately some are never re-edited if errors are found. I am now using the my own inefficient implementation of wiki/Adler32. Have I simply stored the resulting Adler32 bytes in the incorrect order? – SkyPirate Mar 18 '16 at 00:55
Further detail...the Adler32 implementation that I am currently using returns the value of 0xA3570260, which is store as such in the hex file at offset 168 to 171 in the hex file image. Is this correct? – SkyPirate Mar 18 '16 at 01:30
2

As a co-creator of the PNG spec, I can say that the file is definitely not compliant. The Adler-32 check value on the zlib stream is incorrect. (Actually I didn't know that because of anything I co-did -- I ran my own png checker on it.) – Mark Adler Mar 18 '16 at 05:22
Thanks, Mark. I wasn't at a computer where I could check it. – Lee Daniel Crocker Mar 18 '16 at 05:44

A palette base PNG with IDAT that has BTYPE=00 for no compression, now with Adler32 code

2 Answers2

Linked