How to calculate data buffer to zero checksum value with as little data as it's possible

Question

Given this checksum calculation function in C++

int calcCrcPartial(unsigned short* lpdwBlockData, unsigned short dwBlockSizeInBytes, int iInitialCrc)
{
    unsigned short dwBlocksSizeInWords;
    int bIsSigned;
    signed int j;
    signed int i;
    unsigned short dwDataItem;
    bool bIsNegative;

    // number of WORD blocks
    dwBlocksSizeInWords = dwBlockSizeInBytes >> 1;

    for (i = 0; ; ++i)
    {
        if (dwBlocksSizeInWords <= i)
        {
            break;
        }

        dwDataItem = lpdwBlockData[i];

        if (dwDataItem != 0)
        {
            bInvalidCrc = false;
        }

        for (j = 0; j <= 15; ++j)
        {
            bIsSigned = (dwDataItem & 0x8000u) != 0;

            dwDataItem <<= 1;

            bIsNegative = iInitialCrc < 0;

            iInitialCrc <<= 1;
            iInitialCrc += bIsSigned;

            if (bIsNegative)
            {
                iInitialCrc ^= 0x400007u;
            }
        }
    }

    return iInitialCrc;
}

Task:

How to write a function to generate a valid block of data lpdwBlockData (512 bytes) that will make function calcCrcPartial() to return 0 for any given iInitialCrc (calculated from previous calls to this function)?

The CRC is not stored in the block.

The generated block of data (512 bytes) can contain any data.

I have tried to fill the buffer with random data in hope it will hit the 0 checksum value after CRC calculation, but I guess it's not the way...

How to reverse this algorithm and generate valid data to make the calcCrcPartial() on the generated buffer data and provided iInitialCrc to return value 0?

Please show the work you've already written so far, and explain how exactly your function doesn't work or doesn't produce the expected results. You have to show your work first, before asking for help on stackoverflow.com. Stackoverflow.com is not a web site where you can ask someone else to write your program or function from scratch. For more information, see [ask] questions, take the [tour], and read the [help]. — Sam Varshavchik, Nov 06 '19 at 20:00
That's why I'm asking here to get an answer, I have tried to fill it up with random data, but it doesn't work. — Boris Jodorovsky, Nov 06 '19 at 20:21
You did not show any work. All you've shown is a canned, fixed function for which you need to write more code to make it work in a certain way, according to your question's description. Please show all the work you've done so far, and explain exactly why you wrote it in this particular way, and how exactly it "doesn't work". The statement that "it doesn't work" is not very useful. This is like calling your mechanic on the phone, describing the problem with your car as "it doesn't move", and expect the mechanic to know exactly which parts need to be fixed. — Sam Varshavchik, Nov 06 '19 at 20:23
I'm curious as to why you phrased it as a challenge [on codegolf](https://codegolf.stackexchange.com/questions/195392/generate-data-buffer-to-zero-the-crc-check-result) as well. — Borgleader, Nov 06 '19 at 20:46
The parameter names do not correspond to the parameter types. "lp" == "long pointer" (also used for normal pointer), "dw" == "dword" == "uint32_t" == "unsigned int" (for most platforms). — rcgldr, Nov 06 '19 at 21:01
Thanks for your help, but it really doesn't help to solve the problem :), I usually name my variables with my dogs names. Is it wrong? — Boris Jodorovsky, Nov 06 '19 at 21:17

rcgldr · Accepted Answer · 2019-11-08T01:42:27.737

This is not a normal CRC. The initial CRC is cycled left 16 times, then the first short is XOR'ed to the lower 16 bits of the CRC, which is then cycled again 16 times, and the next short is XOR'ed to the lower 16 bits of the CRC. If the data is just 2 shorts, it's the same as cycling the initial CRC 32 times, then XOR'ing the 2 shorts to the cycled CRC. To get the CRC==0, just set the 2 shorts to the initial CRC cycled 32 times. Example code below.

How to calculate data buffer to zero checksum value with as little data as possible

Just need 2 shorts to do this. Set the 2 shorts = 0, calculate CRC, then set the 2 shorts to the calculated CRC. This will result in a CRC of 0 for any initial CRC.

I switched to a table driven version of the checksum function, but the code below also includes a "cleaned up" version of the questions example CRC function.

This code compares the CRC outputs from the questions code, an alternate version, and a table driven version:

#include <iostream>
#include <iomanip>

typedef unsigned short uint16_t;
typedef unsigned int   uint32_t;

uint32_t crctbl[65536];

void gentbl()
{
uint32_t crc;
int i, j;
    for(j = 0; j < 0x10000; j++){
        crc = j<<16;
        for(i = 0; i < 16; i++)
            // assumes twos complement
            crc = (crc<<1)^((0-(crc>>31))&0x400007u);
        crctbl[j] = crc;
    }
}

int calcCrcPartial(unsigned short* lpdwBlockData, unsigned short dwBlockSizeInBytes, int iInitialCrc)
{
    unsigned short dwBlocksSizeInWords;
    int bIsSigned;
    signed int j;
    signed int i;
    unsigned short dwDataItem;
    bool bIsNegative;

    // number of WORD blocks
    dwBlocksSizeInWords = dwBlockSizeInBytes >> 1;

    for (i = 0; ; ++i)
    {
        if (dwBlocksSizeInWords <= i)
        {
            break;
        }

        dwDataItem = lpdwBlockData[i];

//      bInvalidCrc not delcared and not used
//      if (dwDataItem != 0)
//      {
//          bInvalidCrc = false;
//      }

        for (j = 0; j <= 15; ++j)
        {
            bIsSigned = (dwDataItem & 0x8000u) != 0;

            dwDataItem <<= 1;

            bIsNegative = iInitialCrc < 0;

            iInitialCrc <<= 1;
            iInitialCrc += bIsSigned;

            if (bIsNegative)
            {
                iInitialCrc ^= 0x400007u;
            }
        }
    }

    return iInitialCrc;
}

// alternate version of calcCrcPartial
uint32_t calcCrcPartiala(uint16_t* lpwBlockData, uint16_t wBlockSizeInBytes, uint32_t iInitialCrc)
{
int sz = wBlockSizeInBytes >> 1;
int i;
    while(sz--){
        for(i = 0; i < 16; i++)
            // assumes twos complement
            iInitialCrc = (iInitialCrc<<1)^((0-(iInitialCrc>>31))&0x400007u);
        iInitialCrc ^= *lpwBlockData++;
    }
    return iInitialCrc;
}

// table version of calcCrcPartial
uint32_t calcCrcPartialt(uint16_t* lpwBlockData, uint16_t wBlockSizeInBytes, uint32_t iInitialCrc)
{
int sz = wBlockSizeInBytes >> 1;
    while(sz--)
        iInitialCrc = (iInitialCrc<<16)^crctbl[iInitialCrc>>16]^*lpwBlockData++;
    return iInitialCrc;
}

int main()
{
uint16_t data[] = {0x0000, 0x0000};
uint32_t iCrc, oCrc, oCra, oCrt;
    gentbl();
    iCrc = 0x00000000u;
    do{
        oCrc = calcCrcPartial (data, sizeof(data), iCrc);
        oCra = calcCrcPartiala(data, sizeof(data), iCrc);
        oCrt = calcCrcPartiala(data, sizeof(data), iCrc);
        if(oCrc != oCra || oCrc != oCrt){
            std::cout << "mismatch" << std::endl;
            break;}
        if ((iCrc & 0x0ffffffu) == 0)
            std::cout << std::hex << iCrc << std::endl;
    }while(++iCrc != 0x10000000u);

    return 0;
}

This code tests all 4 billion possible initial CRCs.

#include <iostream>
#include <iomanip>

typedef unsigned short uint16_t;
typedef unsigned int   uint32_t;

uint32_t crctbl[65536];

void gentbl()
{
uint32_t crc;
int i, j;
    for(j = 0; j < 0x10000; j++){
        crc = j<<16;
        for(i = 0; i < 16; i++)
            // assumes twos complement
            crc = (crc<<1)^((0-(crc>>31))&0x400007u);
        crctbl[j] = crc;
    }
}

uint32_t calcCrcPartial(uint16_t* lpwBlockData, uint16_t wBlockSizeInBytes, uint32_t iInitialCrc)
{
int sz = wBlockSizeInBytes >> 1;
    while(sz--)
        iInitialCrc = (iInitialCrc<<16)^crctbl[iInitialCrc>>16]^*lpwBlockData++;
    return iInitialCrc;
}

// alternate version of questions code
uint32_t calcCrcPartialx(uint16_t* lpwBlockData, uint16_t wBlockSizeInBytes, uint32_t iInitialCrc)
{
int sz = wBlockSizeInBytes >> 1;
int i;
    while(sz--){
        for(i = 0; i < 16; i++)
            // assumes twos complement
            iInitialCrc = (iInitialCrc<<1)^((0-(iInitialCrc>>31))&0x400007u);
        iInitialCrc ^= *lpwBlockData++;
    }
    return iInitialCrc;
}

int main()
{
uint16_t data[] = {0x0000, 0x0000};
uint32_t iCrc, oCrc;
    gentbl();
    iCrc = 0x00000000u;
    do{
        // oCrc = iCrc cycled 32 times
        data[0] = 0x0000;
        data[1] = 0x0000;
        oCrc = calcCrcPartial(data, 4, iCrc);
        // store oCrc and verify new crc == 0
        data[0] = (oCrc>>16);
        data[1] = (oCrc>> 0);
        oCrc = calcCrcPartial(data, 4, iCrc);
        if (oCrc != 0) {
            std::cout << "error" << std::endl;
            break;
        }
        if ((iCrc & 0xfffffffu) == 0)
            std::cout << std::hex << iCrc << std::endl;
    }while(++iCrc != 0x00000000u);
    return 0;
}

How to calculate data buffer to zero checksum value with as little data as possible

If this means the minimum number of error bits, then in a buffer of 34 shorts, all zero, and initial CRC = 0, 6 bits in specific locations (based on poly and initial CRC) need to be toggled to also produce a CRC = 0:

0x8000, 0x0000, 0x0000, 0x0000, 0x0000,0x0000,0x0000,0x0000,
0x0000, 0x0000, 0x8000, 0x0000, 0x0000,0x0000,0x0000,0x0000,
0x0000, 0x0000, 0x0000, 0x0000, 0x0000,0x0000,0x0000,0x0000,
0x0000, 0x0000, 0x0000, 0x0000, 0x0000,0x0000,0x0020,0x8003,
0x0000, 0x0000

or simple still use the CRC polynomial which only has 5 bits = 1, only 5 words needed:

0x0100, 0x4000, 0x0700, 0x0000, 0x0000

This is how this version of CRC could be used for 512 bytes of data and 4 bytes of CRC:

#include <stdlib.h> // for rand()

// # of shorts in data, not including CRC
#define COUNT 256

int main()
{
uint16_t data[COUNT+2];
uint32_t iCrc, oCrc;
int i;
    gentbl();
    // fill data with psuedo random values
    for(i = 0; i < COUNT; i++)
        data[i] = ((rand()>>4)&0xff)|((rand()<<4)&0xff00);
    iCrc = 0x00000000u;
    do{
        // generate crc
        data[COUNT+0] = 0x0000u;
        data[COUNT+1] = 0x0000u;
        oCrc = calcCrcPartial(data, sizeof(data), iCrc);
        // append crc to data
        data[COUNT+0] = (oCrc>>16);
        data[COUNT+1] = (oCrc>> 0);
        // check crc
        oCrc = calcCrcPartial(data, sizeof(data), iCrc);
        if (oCrc != 0) {
            std::cout << "error" << std::endl;
            break;
        }
        if ((iCrc & 0xfffffu) == 0)
            std::cout << std::hex << iCrc << std::endl;
    }while(++iCrc != 0x01000000u);
    return 0;
}

First of I'd like to thank you for your help. I have tested both solutions. Both have failed. 1. The initial CRC can be anything (it's calculated from previous blocks), if I set iCrc to this value instead of 0x0000000 it fails. 2. Both examples fails at if (oCrc != 0) 3.You are not using calcCrcPartialx() but calcCrcPartial() of your own, can that be a problem? — Boris Jodorovsky, Nov 07 '19 at 21:50
4. The checksum is NOT stored in the blocks or after the block, it's just expected to be 0 afterwards. — Boris Jodorovsky, Nov 07 '19 at 21:57
The iCrc (initial CRC) can be anything (and in your code you are setting it to 0x00000000) and we are looking for the final oCrc of the block with generated data to be 0 (in your code example) for any given iCrc, so for example I want to find the data that will set oCrc to 0 for the iCrc = 0x3932AB (example). It's not a normal case, and the CRC is NOT stored anywhere, the final CRC is expected to be of 0 value, this is the first time I have seen something like this and I really appreciate your help man :) — Boris Jodorovsky, Nov 08 '19 at 01:37
But again the iCrc (initial CRC) is NOT 0, it can be any value, I'm not looking for iCrc, I know its value (from previous blocks CRC calculations), but I don't know what data should I put in the last data buffer (those calculations are made from a file, block by block until the last block that expects to set the final oCrc to 0x00000000) to make the calcCrcPartial(data, 512, 0x123456) return 0, and in your code there is no option to set the iCrc to the given constant value. — Boris Jodorovsky, Nov 08 '19 at 01:55
But your code breaks the loop at the first if (oCrc != 0) why is that? And how to setup iCrc to any given value, because it seems your code doesn't allow that. — Boris Jodorovsky, Nov 08 '19 at 11:15
I'm testing your code and it assumes the CRC is stored within the data, but it's not (it's not required), your code works fine, but it doesn't allow to set the initial CRC to any value say 0xABC. — Boris Jodorovsky, Nov 08 '19 at 11:55
I have tried something like this https://pastebin.com/6PVFWRnh but it doesn't work. The buffer size MUST be 512 bytes, the initial CRC is pre-defined and I need to figure out what data put in those 512 bytes to make the calcCrcPartial() return 0. Can you help me? Thank you again, I would like to learn what am I doing wrong and how to do it properly. — Boris Jodorovsky, Nov 08 '19 at 12:03
I think it works, but I overwrite my source code to test the entire code and I need to re-create it again... I'm so mad ;), will rewrite it and let you know! Anyway thank you again Sir. — Boris Jodorovsky, Nov 08 '19 at 13:15
@BorisJodorovsky - where did you get this code? Is there existing data that uses CRC in the way this code generates the CRC? If not, I would suggest switching to a conventional CRC (xor data first, the cycle CRC). — rcgldr, Nov 08 '19 at 17:45
It's from an old flight simulator. The original files has the data resetting CRC stored as multiple WORDs to reset out the CRC to the 0 value and they are stored at the buffer beginning (they don't fill up the entire buffer). Would it be possible to do it the same way? — Boris Jodorovsky, Nov 08 '19 at 21:27
@BorisJodorovsky - yes, but I'm still not sure about the order of the CRC operations, xor first or cycle first. If the current code matches what is used for the flight simulator, then use it as is. If not, try doing `crc ^= ((uint32_t)**lpwBlockData++)<<16;` first (xor to upper bits), then cycle the crc 16 times. — rcgldr, Nov 09 '19 at 06:23

How to calculate data buffer to zero checksum value with as little data as it's possible

1 Answers1