Difficulty parsing hdf5 compound data type

Question

Using h5dump on the .h5 file, I see the following dataset:

GROUP "T" {
  DATASET "CON" {
    DATATYPE  H5T_COMPOUND {
        H5T_IEEE_F32LE "price";
        H5T_STRING {
           STRSIZE 1;
           STRPAD H5T_STR_NULLTERM;
           CSET H5T_CSET_ASCII;
           CTYPE H5T_C_S1;
        } "label";
        H5T_STD_I64LE "amount";
    }
  }
}

I have created the following data structure in C++:

class RawData
{
public:
    float price;
    char label[2];
    long amount;
};

H5File file2(hdf5Source, H5F_ACC_RDONLY);
DataSet dataset = file2.openDataSet("/T/CON");
size_t size = dataset.getInMemDataSize();
RawData *s = (ExegyRawData*) malloc(size);
CompType type = dataset.getCompType();
dataset.read(s, type);
RawData r = s[0];

When I output RawData members, other than the price field, everything else are not recognizable. Can someone spot what is wrong with the code I have written above?

You might want to try turning the crank backwards - fill in your data structure and try to write it to disk - and see what h5dump shows you. — user888379, Dec 29 '13 at 13:44

score 1 · Accepted Answer · answered Dec 31 '13 at 17:28

There is a distinction between your data representation in memory and on disk. h5dump gives you how things are stored on disk.

For instance, price is a little-endian 32 bit floating-point number, but if your computer is big-endian, the library will convert it for you when reading and the memory representation will be H5T_IEEE_F32BE.

Another issue might be padding of structs. Chances are that your structure will be aligned such that the offset in bytes of your members are 0, 4 and 8. But to save disk space, the library might prefer the more compact alignment with offsets 0, 4 and 6.

Solution: Create a proper H5::CompType corresponding to your struct.

CompType type(sizeof(RawData));
type.insertMember("price", HOFFSET(RawData, price), PredType::NATIVE_FLOAT);
type.insertMember("label", HOFFSET(RawData, label), StrType(0, 2));
type.insertMember("amount", HOFFSET(RawData, amount), PredType::NATIVE_LONG);

Difficulty parsing hdf5 compound data type

1 Answers1