Mismatching MD5 hash using QCryptographicHash on CentOS and Win7

Question

Preamble

I'm working on a project that requires FTP file transfer between two machines, a client and a server. I would like to verify the integrity of the file after transfer by comparing an MD5 checksum generated on the client with one generated on the server. I wrote a command line program using Qt's QCryptographicHash class to do this. Minus error checking, this is the code in its entirety:

Minimal Working Example

#include <QCoreApplication>
#include <QByteArray>
#include <QFile>
#include <QCryptographicHash> 
#include <QTextStream>
#include <QStringList>

int main (int argc, char* argv[]) {
    QCoreApplication app(argc, argv);
    QTextStream cout(stdout, QIODevice::WriteOnly);
    QStringList arglist = app.arguments();

    QFile file(arglist.at(1));
    if(!file.open(QIODevice::ReadOnly)){
        /*error checking*/
    }


    QCryptographicHash cryptoHash(QCryptographicHash::Md5);
    while(!file.atEnd()){
        cryptoHash.addData(file.readLine());
    }

    QByteArray hashByteArray = cryptoHash.result();

    cout << hashByteArray.toHex() <<endl;
    file.close();
    return 0;
}

The Problem

On any machine, this code will give a repeatable MD5 hash of any input file. However, different hashes are being generated for the same file when hashed on my client than when hashed on my server. I've been fighting with this awhile, and just can't seem to determine why these hashes aren't matching. My last thought is that it may be an issue related to architecture, as discussed in this SO post. However, in my case, I'm using Qt's implementation of MD5 hashing, so I want to make sure there isn't something else I'm missing before I either abandon Qt's approach or tinker with their source (which is here md5.h and here md5.cpp, BTW).

Machine specifics. Server: CentOS 5.8 64-bit. Client: Win7 64-bit, but on Windows this application was compiled using a 32-bit build of Qt, and is a 32-bit application.

As an aside, I intended on distributing both 32-bit and 64-bit versions of my application. How can I reconcile this issue if it is architecture related and I'm stuck using Qt's source?

EDIT 1: I forgot to mention that I'm transferring all files in Binary mode through FTP and the file sizes do match on the client and the server. I was initially convinced this was a line ending issue, but it doesn't appear to be the case since I'm encountering the issue even in Binary mode FTP transfer.

Your server and client are platforms that use different text file line ending conventions. Are you transferring your file in binary mode? (You should be.) Also, check the file size on the server and on the client. If the sizes differ, then the MD5 hash is (almost) guaranteed not to match, and it's probably a line ending problem. — Greg Hewgill, Oct 30 '12 at 23:25
Sorry, I should've pointed out that I am transferring in Binary mode and, yes, the file sizes are the same. — Eli Hooten, Oct 31 '12 at 00:54
Still seems like a line ending problem is the most likely scenario. readLine() munges "\r\n" into "\n" on Windows (only), according to the documentation, so this could account for the difference. Try replacing `file.readLine()` with `file.read(4096)`. — rohanpm, Oct 31 '12 at 02:40
@rohanpm Ran some tests with your approach rohanpm. It looks like any file I pass in that has one line only results in a matching checksum, anything larger than one line results in an inaccurate checksum. Now I'm leaning toward a line ending problem again... — Eli Hooten, Oct 31 '12 at 03:29
Ah it was line endings, after I'd convinced myself it wasn't. In my haste to prepare the minimal example, I misrepresented how I was opening the file. I was actually doing file.open(QIODevice::ReadOnly | QIODevice::Text). Which of course, led Qt to munge the line endings. The problem? This was only in place on my client side code, not the server side hash calculation. Fixed now. Thanks guys. — Eli Hooten, Oct 31 '12 at 04:11

score 2 · Answer 1 · answered Feb 10 '16 at 08:58

2

No need to work on content part unless you already have it.

QCryptographicHash hash(QCryptographicHash::Md5);     
QFile file(path);
 if (file.open(QIODevice::ReadOnly))
    {

       if ( hash.addData(&file) )
           return hash.result().toHex();


    }

answered Feb 10 '16 at 08:58

Yash

6,644
4
36
26

This is the right answer. Even works on 32 bit systems. The others do not. – TimZaman Dec 30 '16 at 15:21

score 0 · Accepted Answer · answered Oct 31 '12 at 07:13

0

You should not use readline a better approach is to read it in a byte array and then calculate the hash like this

QFile file("/home/opc0de/somefile.dat");

    if (file.open(QIODevice::ReadOnly)) {
        QByteArray fileData = file.readAll();
        QByteArray hashData = QCryptographicHash::hash(fileData, QCryptographicHash::Md5);

        qDebug() << hashData.toHex();
    }

Hope it helps.

answered Oct 31 '12 at 07:13

opc0de

11,557
14
94
187

This is the exact solution I arrived at last night. You beat me to the answer,though, so the check mark is yours. Thanks. – Eli Hooten Oct 31 '12 at 15:05

Mismatching MD5 hash using QCryptographicHash on CentOS and Win7

Preamble

Minimal Working Example

The Problem

2 Answers2