Converting data types to add to QByteArray for writing raw data to file

Question

I have some data types that I am trying to write to to a raw data file. I'm not using QDataStream because that writes some extra information about the data, like the length and order of data. I simply want a file that is only the bytes that I write, and will only be interpreted by someone who knows the correct order and size of the data types being written.

I am using QFile, with the write method qint64 QIODevice::write(const QByteArray &byteArray).

I have a few uint8_t, uint32_t and floats to write to the file. How can I transform this data into a QByteArray with no extra bytes?

Here is what I have so far:

void FileWriter::writeData(uint8_t status, uint8_t channel, uint32_t ticks, float source0, float source1){

    QByteArray dataToWrite;

    //some code to add the paramters to the dataToWrite array

    this->file.write(dataToWrite);
}

A regular old style cast to char gives me an implicit conversion changes signedness error, so that wouldn't store the value correctly.

what is the proper way to copies these values into a QByteArray so I can write it to file?

Answer 1

Generally you want to serialize your data streams while keeping note of the endianness of the data.

uint8_t variables can be converted to bytes with a simple static_cast. For larger integer types, you need to keep track of the endianness of the data, and push it byte by byte.

float data types are a little bit trickier. You first need to represent the bits of the float as an integral value (uint32_t), then serialize from there. An example can be shown below:

QByteArray data;

//serialize a uint8_t
data.push_back(static_cast<char>(status));

//serialize a uint32_t, little-endian
data.push_back(static_cast<char>((ticks) & 0xFF); //lowest-order byte
data.push_back(static_cast<char>((ticks >> 8) & 0xFF));
data.push_back(static_cast<char>((ticks >> 16) & 0xFF));
data.push_back(static_cast<char>((ticks >> 24) & 0xFF)); //highest-order byte

//serialize a float, by first representing the bits as a uint32_t: 

static_assert(sizeof(float) == sizeof(uint32_t), "Floats should be 4 bytes");
uint32_t rep;
std::memcpy(&rep, &source0, sizeof(float)); //using memcpy here so that we don't violate strict aliasing. 
data.push_back(static_cast<char>((rep) & 0xFF);
data.push_back(static_cast<char>((rep >> 8) & 0xFF));
data.push_back(static_cast<char>((rep >> 16) & 0xFF));
data.push_back(static_cast<char>((rep >> 24) & 0xFF));

this->file.write(data);

A few things to keep in mind:

1. The endianness of your data is important. Not all systems have the same endianness so you need to be explicit in your documentation about which way the bytes are oriented.

2. Some people try to convert the bits of a float to a uint32_t by writing something like uint32_t rep = *reinterpret_cast<uint32_t*>(&source0);. Don't do this -- it is a violation of the Strict Aliasing Rule.

3. It may be a good idea to serialize a version number as the first part of your file so that you can make future changes to the data format and have it be backwards-compatible.

4. A lot of this manual bit-packing can be written into template functions that will do it automatically. This is left as an exercise to the reader.

5. If performance is important, you should figure out how large the data will be and reserve() the space in the byte array before pushing -- this will save the array from having to reallocate and grow over and over.