Calculate File CheckSum value from Java

Question

I am trying to write a code to generate the same Checksum value as generated by UNIX cksum command.

When I run my code and command on the same file, I am getting different values.

What is wrong with my code?

package cksum;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.security.DigestInputStream;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.zip.Adler32;
import java.util.zip.CRC32;
import java.util.zip.CheckedInputStream;

public class Cksum1 {
    public static void main(String[] args) throws NoSuchAlgorithmException, IOException {
        //MessageDigest md = MessageDigest.getInstance("MD5");
        String File = "C:\\Users\\admin\\Desktop\\logback XMLs\\STG_logback_Debug_All.xml";
        //String File = args[0];
        doChecksum32(File);
    }

    private static void doChecksum32(String fileName) {
        try {
            CheckedInputStream cis = null;
            long fileSize = 0;
            try {
                // Computer CRC32 checksum
                cis = new CheckedInputStream(
                        new FileInputStream(fileName), new CRC32());
                fileSize = new File(fileName).length();
            } catch (FileNotFoundException e) {
                System.err.println("File not found.");
                System.exit(1);
            }
            byte[] buf = new byte[128];
            while (cis.read(buf) >= 0) {
            }
            long checksum = cis.getChecksum().getValue();
            System.out.println(checksum + " " + fileSize + " " + fileName);
        } catch (IOException e) {
            e.printStackTrace();
            System.exit(1);
        }
    }

}

Java Output:

run: 1478491164 8335 C:\Users\admin\Desktop\logback XMLs\STG_logback_Debug_All.xml BUILD SUCCESSFUL (total time: 0 seconds)

Unix output:

[tibroot@myserver ~]$ cksum ./STG_logback_Debug_All.xml 1715323814 8335 ./STG_logback_Debug_All.xml

Possible duplicate of [Java compatible cksum function](https://stackoverflow.com/questions/7746476/java-compatible-cksum-function) — MS90, Mar 29 '19 at 22:09
@farmer1992 explained it here why is that, but you have a link with all chksums in palacsint answer. — MS90, Mar 29 '19 at 22:10

score 2 · Answer 1 · answered Mar 29 '19 at 22:05

2

polynomial for cksum is 0x04C11DB7

while jdk use x^32 + x^26 ...

see also https://en.wikipedia.org/wiki/Cyclic_redundancy_check

answered Mar 29 '19 at 22:05

farmer1992

7,816
3
30
26

So, Java code will never give the same value as Unix? – AKPuvvada Apr 01 '19 at 14:05
You can choose other crc32 library whose aglo is same as cksum – farmer1992 Apr 02 '19 at 22:58
I googled maybe you may want to try this https://github.com/snksoft/java-crc/blob/master/src/main/java/com/github/snksoft/crc/CRC.java#L135 – farmer1992 Apr 04 '19 at 10:32
I tried that and also check different CRC Java implementations. None of them returned the same value as the Unix server. – AKPuvvada Apr 08 '19 at 17:45
Just wondering, I am running my Java code on my PC (Win 10) and running the command on Oracle Free Linux OS - does this matter? – AKPuvvada Apr 08 '19 at 17:46
No JVM will yield same result on every platform – farmer1992 Apr 09 '19 at 22:03
You are checking an XML file. Make sure that you binary copy it between Windows to Linux. Some copy tools feel free to change the line ending bytes inside the file; this wil break any checksum. – Matthias Ronge Dec 02 '19 at 07:33

score 0 · Answer 2 · answered Jan 24 '20 at 01:58

Sorry, this a partial answer, but I demonstrate getting matching results from cksum -o 3 and Java.

I am not certain of the default algorithm implementation of cksum. However, "algorithm 3" of cksum is exactly the same as ava.util.zip.CRC32. In other words, if I do: cksum -o 3 somefile - I get get the same result in Java with:

 static long getCRC32(String filePath) throws IOException {
    CRC32 crc = new CRC32(); 
    ByteBuffer buffer = ByteBuffer.allocate(1024);

    int len = 0;
    try(SeekableByteChannel input = Files.newByteChannel(Paths.get(filePath), StandardOpenOption.READ)) {
      while((len = input.read(buffer)) > 0) {
        buffer.flip();
        crc.update(buffer.array(), 0, len);
      }
    }

    return crc.getValue();   
  }

Calculate File CheckSum value from Java

2 Answers2