17

For the past 4 hours I've been studying the CRC algorithm. I'm pretty sure I got the hang of it already.

I'm trying to write a png encoder, and I don't wish to use external libraries for the CRC calculation, nor for the png encoding itself.

My program has been able to get the same CRC's as the examples on tutorials. Like on Wikipedia: enter image description here

Using the same polynomial and message as in the example, I was able to produce the same result in both of the cases. I was able to do this for several other examples as well.

However, I can't seem to properly calculate the CRC of png files. I tested this by creating a blank, one pixel big .png file in paint, and using it's CRC as a comparision. I copied the data (and chunk name) from the IDAT chunk of the png (which the CRC is calculated from), and calculated it's CRC using the polynomial provided in the png specification.

The polynomial provided in the png specification is the following:

x32 + x26 + x23 + x22 + x16 + x12 + x11 + x10 + x8 + x7 + x5 + x4 + x2 + x + 1

Which should translate to:

1 00000100 11000001 00011101 10110111

Using that polynomial, I tried to get the CRC of the following data:

01001001 01000100 01000001 01010100
00011000 01010111 01100011 11101000
11101100 11101100 00000100 00000000
00000011 00111010 00000001 10011100

This is what I get:

01011111 11000101 01100001 01101000 (MSB First)
10111011 00010011 00101010 11001100 (LSB First)

This is what is the actual CRC:

11111010 00010110 10110110 11110111

I'm not exactly sure how to fix this, but my guess would be I'm doing this part from the specification wrong:

In PNG, the 32-bit CRC is initialized to all 1's, and then the data from each byte is processed from the least significant bit (1) to the most significant bit (128). After all the data bytes are processed, the CRC is inverted (its ones complement is taken). This value is transmitted (stored in the datastream) MSB first. For the purpose of separating into bytes and ordering, the least significant bit of the 32-bit CRC is defined to be the coefficient of the x31 term.

I'm not completely sure I can understand all of that.

Also, here is the code I use to get the CRC:

 public BitArray GetCRC(BitArray data)
    {
        // Prepare the divident; Append the proper amount of zeros to the end
        BitArray divident = new BitArray(data.Length + polynom.Length - 1);
        for (int i = 0; i < divident.Length; i++)
        {
            if (i < data.Length)
            {
                divident[i] = data[i];
            }
            else
            {
                divident[i] = false;
            }
        }

        // Calculate CRC
        for (int i = 0; i < divident.Length - polynom.Length + 1; i++)
        {
            if (divident[i] && polynom[0])
            {
                for (int j = 0; j < polynom.Length; j++)
                {
                    if ((divident[i + j] && polynom[j]) || (!divident[i + j] && !polynom[j]))
                    {
                        divident[i + j] = false;
                    }
                    else
                    {
                        divident[i + j] = true;
                    }
                }
            }
        }

        // Strip the CRC off the divident
        BitArray crc = new BitArray(polynom.Length - 1);
        for (int i = data.Length, j = 0; i < divident.Length; i++, j++)
        {
            crc[j] = divident[i];
        }
        return crc;
    }

So, how do I fix this to match the PNG specification?

MythicManiac
  • 445
  • 1
  • 4
  • 11
  • You need to read [this tutorial](http://www.ross.net/crc/download/crc_v3.txt). First off, this is not the place to get your code reviewed. It is wrong. Second, you are approaching how to calculate a CRC in entirely the wrong way. You should be using the exclusive-or operation, not `&& || (! && !)`, and over multiple bits with one operation. Third, even if you had gotten your code working, you are not pre and post processing the CRC by inverting it. – Mark Adler Jun 06 '14 at 15:18
  • 1
    I'm aware of this not being a place to get my code reviewed, however, I thought it'd probably help with my question if I included the code. I'm not going over multiple bits with one operation yet, because I want to get the very basics working before I start to optimize my code to be faster. I want to understand the code, not just copy paste it from some place on the internet. Also, I think I made it pretty clear my code IS working, or at least is working on the examples I found on the guides, the tutorial you linked being one of them. – MythicManiac Jun 06 '14 at 16:56
  • Your code is not working, in that neither of the provided results matches the expected "pure" CRC-32 of the provided data. By "pure" I mean without the pre and post-processing of the CRC. – Mark Adler Jun 06 '14 at 23:23
  • The question I'm asking is basically what is this pre and post process exactly? Either I didn't read the tutorial far enough (I stopped at the implementation, as mine wasn't working), or it isn't explained there. – MythicManiac Jun 06 '14 at 23:56
  • Invert every bit in the CRC. Then execute the CRC process you are attempting with the proper polynomial and bit ordering. Then when done, invert every bit in the CRC again. Those two inversions are the pre and post processing. A pure CRC will be unchanged by a string of zeros of any length. A pre and post processed CRC _will_ be changed by a string of zeros, and the result will depend on how many zeros. – Mark Adler Jun 07 '14 at 00:03
  • I take it this means I have to do this to match the specification: 1: Initialize CRC to all 1's. 2: Process the CRC using the LSB first version of the polynomial. 3: Flip the resulting CRC around so it's MSB first. If this is incorrect, you should see why I don't get the result correctly. I appreciate all of your help, though. – MythicManiac Jun 07 '14 at 00:14
  • Yes on 1. I'm not sure what you mean by 2. The polynomial does need to be reversed, so that the x^31 term is in the least significant bit. Having done that, you do not do 3. You also need to invert each bit (0->1, 1->0) at the end. – Mark Adler Jun 07 '14 at 16:43
  • @MarkAdler what do you mean by "invert every bit of the CRC"? do you mean invert the data you're feeding into the CRC algorithm, or invert the polynomial, or something else entirely? and what do you mean by "MSB"? most significant BYTE, or most significant BIT? – MarcusJ Oct 26 '17 at 05:25
  • 2
    @MarcusJ "Invert every bit of the CRC" means invert every bit of the CRC. The CRC is the result of the computation. It is not the data, nor the polynomial. – Mark Adler Oct 26 '17 at 13:51
  • 1
    Here MSB is most significant bit. CRCs are always about bits. They are agnostic to the existence of bytes. – Mark Adler Oct 26 '17 at 13:52

1 Answers1

16

You can find a complete implementation of the CRC calculation (and PNG encoding in general) in this public domain code:

static uint[] crcTable;

// Stores a running CRC (initialized with the CRC of "IDAT" string). When
// you write this to the PNG, write as a big-endian value
static uint idatCrc = Crc32(new byte[] { (byte)'I', (byte)'D', (byte)'A', (byte)'T' }, 0, 4, 0);

// Call this function with the compressed image bytes, 
// passing in idatCrc as the last parameter
private static uint Crc32(byte[] stream, int offset, int length, uint crc)
{
    uint c;
    if(crcTable==null){
        crcTable=new uint[256];
        for(uint n=0;n<=255;n++){
            c = n;
            for(var k=0;k<=7;k++){
                if((c & 1) == 1)
                    c = 0xEDB88320^((c>>1)&0x7FFFFFFF);
                else
                    c = ((c>>1)&0x7FFFFFFF);
            }
            crcTable[n] = c;
        }
    }
    c = crc^0xffffffff;
    var endOffset=offset+length;
    for(var i=offset;i<endOffset;i++){
        c = crcTable[(c^stream[i]) & 255]^((c>>8)&0xFFFFFF);
    }
    return c^0xffffffff;
}

1 https://web.archive.org/web/20150825201508/http://upokecenter.dreamhosters.com/articles/png-image-encoder-in-c/

EricLaw
  • 56,563
  • 7
  • 151
  • 196
  • 3
    One can test this with the `IEND` chunk that should always produce the bytes `0xae 0x42 0x60 0x82`, since it never changes its name, nor does it ever have any payload. Look at your existing files: they should all end with those bytes. – AmigoJack Jun 09 '21 at 11:06