Stack Overflow
Stack Overflow

Questions tagged [crc32]

A cyclic redundancy check (CRC) is an error-detecting code designed to detect accidental changes to raw computer data, and is commonly used in digital networks. (wiki) A CRC32 algorithm typically takes in a file stream or character array and calculates an unsigned long codeword from the input. One can transmit this codeword and re-calculate it on the receiver end, then compare it to the transmitted one to detect an error.

A cyclic redundancy check (CRC) is an error-detecting code commonly used in digital networks and storage devices to detect accidental changes to raw data. Blocks of data entering these systems get a short check value attached, based on the remainder of a polynomial division of their contents; on retrieval the calculation is repeated, and corrective action can be taken against presumed data corruption if the check values do not match. wikipedia

The most commonly used polynomial lengths are:

  • CRC-8: 9 bits
  • CRC-16: 17 bits
  • CRC-32: 33 bits
  • CRC-64: 65 bits

A truly excellent tutorial on CRC's is Ross Williams' "Painless Guide to CRC Detection Algorithms", which can also be found here, here, here, here, and here.

570 questions
10
votes
3 answers

Is there a LINUX command line tool for CRC32C (Castagnoli)

I need to verify data using CRC32C, the Castagnoli variant of CRC32 calculation. I cannot find a self-contained command line utility anywhere that will calculate this to verify the values in question. CRC32? MD5? SHA? Check. CRC32C? You lose. I need…
Mark Gerolimatos
  • 2,424
  • 1
  • 23
  • 33
10
votes
2 answers

Checksumming: CRC or hash?

Performance and security considerations aside, and assuming a hash function with a perfect avalanche effect, which should I use for checksumming blocks of data: CRC32 or hash truncated to N bytes? I.e. which will have a smaller probability to miss…
ayurchen
  • 386
  • 3
  • 9
9
votes
1 answer

Why does BCL GZipStream (with StreamReader) not reliably detect Data Errors with CRC32?

The the other day I ran into the question GZipStream doesn't detect corrupt data (even CRC32 passes)? (Of which this might very well be a "duplicate", I have mixed feelings on the subject. I was also the one who added the CRC32 to the title, but in…
user166390
9
votes
2 answers

_mm_crc32_u64 poorly defined

Why in the world was _mm_crc32_u64(...) defined like this? unsigned int64 _mm_crc32_u64( unsigned __int64 crc, unsigned __int64 v ); The "crc32" instruction always accumulates a 32-bit CRC, never a 64-bit CRC (It is, after all, CRC32 not CRC64). …
David I. McIntosh
  • 2,038
  • 4
  • 23
  • 45
9
votes
2 answers

Is CRC32 really so bad for file integrity check?

Of course that MD5 is better then CRC32, SHA1 is better then MD5 and so on... But also they are also much slower then CRC32. Right know, I am thinking about how to check consistency of being transfered file and CRC32 is fastest option. I haven't…
Matt Warrick
  • 1,385
  • 1
  • 13
  • 22
8
votes
3 answers

Java compatible cksum function

Is there any library/code in Java to calculate the 32-bit CRC of a stream of bytes in a way thats consistent with the cksum command in unix ?
Kowshik
  • 1,541
  • 3
  • 17
  • 25
8
votes
2 answers

Python find CRC32 of string

I tried to get crc32 of a string data type variable but getting the following error. >>> message='hello world!' >>> import binascii >>> binascii.crc32(message) Traceback (most recent call last): File "", line 1, in TypeError: a…
Dinesh
  • 1,825
  • 5
  • 31
  • 40
8
votes
1 answer

How to properly use carry-less multiplication assembly (PCLMULQDQ) in zlib CRC32?

I've recently been playing around with CloudFlare's optimized zlib, and the results are really quite impressive. Unfortunately, they seem to have assumed development of zlib was abandoned, and their fork broke away. I was eventually able to manually…
Geoff Nixon
  • 4,697
  • 2
  • 28
  • 34
8
votes
2 answers

Expected collisions for perfect 32bit crc

I'm trying to determine how my crc compares to an "ideal" 32bit crc. So I ran my crc over 1 million completely random samples of data and collected the amount of collisions, I want to compare this number to the number of collisions I could expect…
Tristan
  • 3,845
  • 5
  • 35
  • 58
8
votes
2 answers

How can i receive the wrong Ethernet frames and disable the CRC/FCS calcul?

I generate a traffic between two PCs running Linux (by sending Ethernet frames), the goal of this is to capture some errors frames. The problem is when the Phy layer detect an error on a frame (if the CRC or FCS is not valid) the frame is dropped…
A2maridz
  • 341
  • 1
  • 4
  • 11
7
votes
5 answers

CRC32+Size vs MD5/SHA1

We have a storage of files and the storage uniquely identifies a file on the basis of size appended to crc32. I wanted to know if this checksum ( crc32 + size ) would be good enough for identifying files or should we consider some other hashing…
Rajiv
  • 545
  • 1
  • 6
  • 12
7
votes
1 answer

Calculate CRC32, MD5 and SHA1 of zip content without decompression in Python

I need to calculate the CRC32, MD5 and SHA1 of the content of zip files without decompressing them. So far I found out how to calculate these for the zip files itself, e.g.: CRC32: import zlib zip_name = "test.zip" def Crc32Hasher(file_path): …
paradadf
  • 123
  • 1
  • 7
7
votes
3 answers

Difficulty comparing generated and google cloud storage provided CRC32c checksums

I am attemptting to get a CRC32c checksum on my local file so I can compare it to the blob.crc32c provided by the gcloud library. Google says I should be using the crcmod module in order to actually calculate CRC32c hashes of my data.…
Stephen Cagle
  • 14,124
  • 16
  • 55
  • 86
7
votes
1 answer

Is the uniqueness of CRC-32-hashes sufficient to uniquely identify strings containing filenames?

I have sorted lists of filenames concatenated to strings and want to identify each such string by a unique checksum. The size of these strings is a minimum of 100 bytes, a maximum of 4000 bytes, and an average of 1000 bytes. The total number of…
MCH
  • 415
  • 3
  • 11
7
votes
1 answer

PHP crc32() only numbers

I have a MD5 hash: 10f86782177490f2ac970b8dc4c51014 http://www.fileformat.info/tool/hash.htm?text=10f86782177490f2ac970b8dc4c51014 Result: c74e16d9 but PHP: crc32('10f86782177490f2ac970b8dc4c51014'); Result: -951183655 I don't understand!
B11002
  • 95
  • 1
  • 2
  • 5
1 2
3
…
37 38