Is there a LINUX command line tool for CRC32C (Castagnoli)

Question

I need to verify data using CRC32C, the Castagnoli variant of CRC32 calculation. I cannot find a self-contained command line utility anywhere that will calculate this to verify the values in question. CRC32? MD5? SHA? Check. CRC32C? You lose.

I need a program. Has to be self contained (no non-standard PERL or Python modules that require installation...standard ones are okay). Probably no Java, as I cannot guarantee a Java installation. Ruby is right out.

Clever "gzip" or "tar" tricks are fine. Needs to run on OS-X, Linux, and maybe Android.

I could indeed write one (the code is in the RFC), but c'mon...that's sort of like

"hmmm, my car uses synthetic oil...time for an oil change, let me go to the local refinery and make a few quarts"

or

"Uncle Bill wants a PB&J sandwich, but has a gluten allergy...let me go get some rice flour and make gluten-free bread".

I found some libraries to calculate CRC32Cs, but no programs. Grumble.

https://developers.google.com/storage/docs/composite-objects?hl=ja

Trivia: SSE4.2 has an instruction for this http://en.wikipedia.org/wiki/SSE4#SSE4.2 (mind blown) — sehe, Jun 24 '14 at 23:29
A online CRC32C calculator is available at https://www.scadacore.com/tools/programming-calculators/online-checksum-calculator/ — Encombe, Feb 27 '19 at 17:10

Travis Hobrla · Answer 1 · 2017-11-28T19:35:14.733

13

Starting with 4.4, gsutil supports a 'hash' command that can compute hashes like so:

gsutil hash filename

It will output CRC32C and MD5 by default.

edited Nov 28 '17 at 19:35

answered Jun 24 '14 at 23:57

Travis Hobrla

5,411
23
22

Installing the native crcmod is a bit ugly because it requires to uninstall the existing one. https://cloud.google.com/storage/docs/gsutil/addlhelp/CRC32CandInstallingcrcmod#debian-and-ubuntu – pablo Aug 04 '18 at 17:49
1

First `pip install gsutil`. – Smeterlink Jul 17 '21 at 20:42

score 2 · Answer 2 · edited May 23 '17 at 12:20

2

So long as you can compile C code, you can find the program you're looking for in this stackoverflow answer. It will also use the Intel crc32 instruction if available.

edited May 23 '17 at 12:20

Community

1
1

answered Jun 25 '14 at 05:40

Mark Adler

101,978
13
118
158

Is there an easy to use command line tool that calculate crc32c of large files by reading chunks? – pablo Aug 04 '18 at 17:48
If you compile the program at that link with `TEST` defined, then you have your command-line tool. – Mark Adler Aug 04 '18 at 18:05
1

It's working with "gcc -D TEST crc32c.c -o crc32c -lpthread" and "cat bigfile | ./crc32c" Thanks. – pablo Aug 05 '18 at 11:12
1

Or simply `./crc32c < bigfile`. – Mark Adler Aug 05 '18 at 15:12

score 1 · Answer 3 · answered Jun 24 '14 at 23:56

1

gsutil will include a new "hash" command in the next release (gsutil 4.4) - which should be out pretty soon.

answered Jun 24 '14 at 23:56

Mike Schwartz

11,511
1
33
36

Is there a LINUX command line tool for CRC32C (Castagnoli)

3 Answers3