1

Qt/C++ program has a function which writes an objects data (_token) to a file as follows:

    QFile f( _tokenFile);

    if (!f.open( QIODevice::WriteOnly)) {
        qDebug() << "Unable to open token file for writing" << f.errorString();
        return false;
    }

    QByteArray tokenBa;
    QDataStream ds( &tokenBa, QIODevice::WriteOnly);
    ds << _token;

    tokenBa = qCompress( tokenBa);

    f.write( tokenBa);
    f.close();

The _token is an instance of the following struct:

struct Token {
    QString accessToken;
    QString refreshToken;
    QString clientSecret;
    QString authCode;
    QTime expiryTime;


    enum AuthState {
        A_Invalid,
        A_RequestAuth,
        A_Authenticated,
    };

    AuthState state;

    Token() : state( A_Invalid) {}

    bool isValid() const {
        if (accessToken.isEmpty() ||
            refreshToken.isEmpty()) {
            return false;
        }
        return true;
    }

    void inValidate() {
        accessToken.clear();
        refreshToken.clear();
        clientSecret.clear();
        authCode.clear();
        expiryTime = QTime();
    }

    void cleanUp() {
        accessToken.clear();
        refreshToken.clear();
        clientSecret.clear();
        authCode.clear();
        expiryTime = QTime();
    }
};

When the file is saved it has 4 extra bytes at the start which render the file as a invalid zlib file.


0000000 0000 5e01 9c78 904d 4f5d 5082 871c fa9f
0000020 353e 25cd 6975 2c2f d563 4c2c 62b8 cad1

We can see above bytes 5-6 are 9C 78 which is the zlib signature, but the 4 bytes before these are the issue.

To check the compressed data is correct I do the following:

dd if=file.token bs=1 skip=4 | openssl zlib -d

And this produces the expected result (for testing).

The problem is in the application reading this data back into the data object:

    QFile f( _tokenFile);

    if (!f.exists()) {
        qDebug() << "Token file doesn't exist"  << f.fileName();
        return false;
    }

    if (!f.open( QIODevice::ReadOnly)) {
        qDebug() << "Unable to open token file for reading" << f.errorString();
        return false;
    }

    QByteArray tokenBa = f.readAll();
    f.close();

    if (tokenBa.isEmpty()) {
        qDebug() << "Token file is empty.";
        return false;
    }

    tokenBa = qUncompress( tokenBa);

    QDataStream ds( &tokenBa, QIODevice::ReadOnly);
    ds >> _token;

This returns null - because of the leading 4 extraneous bytes. I could put some code in to skip these 4 leading bytes, but how do I know it will always be 4 bytes? I'd like to instead have certainly that the files data is all zlib compressed.

My question is how to avoid those bytes being saved in the first place so that on re-read the format is known to be zlib type?

TenG
  • 3,843
  • 2
  • 25
  • 42

1 Answers1

1

You can't avoid them since they're needed for qUncompress later on:

Note: If you want to use this function to uncompress external data that was compressed using zlib, you first need to prepend a four byte header to the byte array containing the data. The header must contain the expected length (in bytes) of the uncompressed data, expressed as an unsigned, big-endian, 32-bit integer.

chehrlic
  • 913
  • 1
  • 5
  • 5
  • Thanks. So in my example the first 4 bytes `0000 5e01` should do precisely that, which then is puzzling why the uncompress fails. – TenG Mar 05 '21 at 11:22
  • 0x5e01 is 24065 - therefore your file should be 24069 bytes big and also the QByteArray you read in - is this the case? – chehrlic Mar 05 '21 at 15:46
  • No. The actual size is about 240 bytes. Maybe the issue is int the save of the file. I'll do some more debugging and check. – TenG Mar 05 '21 at 17:50
  • Then add a debug statement to print out the size of tokenBA while writing the file. And maybe also output the first 4 bytes of the bytearray there to see if they contain the expected bytes. – chehrlic Mar 06 '21 at 10:49
  • not really .. best way to write such binary streams is `QFile::write( QByteArray)` not `QdataStream` – Mohammad Kanan Jul 29 '22 at 18:55