Parse binary into a structure dynamically

Question

I have to read a binary file which is usually 266 bytes but sometimes can have additional dynamic fields. The binary fields are positional and when they are not there they are inserted anyway at 00 with the declared length. The only dynamic fields are the last two (defined in the code with two vectors). I would like to do something like this:

 #pragma pack(push,1)
 typedef struct binary_structure{
    unsigned short  len;
    uint8_t   system;
    uint8_t   vers;
    unsigned char  data1                [6];
    unsigned char  data2                [4];
    unsigned short data3;
    unsigned char  data4                [6];
    unsigned char  data5                [4];
    unsigned short data6;
    unsigned short data_ses;
    unsigned char  data_list            [40];
    unsigned char  timestamp_1          [4];
    unsigned char  timestamp_2          [4];
    int            data_byte_1;
    int            data_byte_2;
    int            data_pkt_1;
    int            data_pkt_2;
    unsigned char  data_file_trace      [50];
    unsigned char  data_file_2          [40];
    unsigned char  data_username        [30];
    unsigned char  data_7               [20];
    unsigned short data_num;
    int            data_index;
    unsigned char  data_value           [20];
    unsigned short data_url_index;
    unsigned short data_len_url;
    std::vector <unsigned char> data_url_value(data_len_url);
    unsigned char   data_info_t          [2];
    unsigned short  data_info_len;
    std::vector <unsigned char> data_info_value(data_info_len);
 }binary_structure_T, *sBinary_structure_t;
 #pragma pack(pop)

As you can see I'm trying to initialize the two vectors data_info_value and data_url_value dynamically only if they are present in the binary. Usually there are none and the respective data_len_url and data_info_len fields are set to 00 00. If there are, their lengths are defined in the two fields data_info_len and data_len_url, so When these fields have the two respective lengths I would like to use them as a value to define the dimensions of the respective **data_info_value **and data_url_value vectors.

The binary file is parsed into the above structure as follows:

int main(int argc, char *argv[] {
  sBinary_structure_t binary = (sBinary_structure_t)(file_buffer.data());
}

Is it possible to do it dynamically like above or do I have to do it manually?

Certainly, write a constructor for `sBinary_structure_t` or a cast operator for whatever `file_lidas_buffer.data()` returns. But it's all manual mate, just a question of how you organize the code. — user207421, Jun 07 '23 at 08:21
You can't load or save a structure with `std::vector` inside it. And you can't do inline initialization using parentheses, or using object members. — Some programmer dude, Jun 07 '23 at 08:23
On a different note, you don't need `typedef` for structures or classes in C++. — Some programmer dude, Jun 07 '23 at 08:23
@user207421 It's not all manual, in this case it's possible if the size of the track coincides with that of the structure but my question was different. Because there is a difference between doing as I wrote above and manually reading the binary by loading the individual values into the structure fields. — XvaneD, Jun 07 '23 at 08:25
@Someprogrammerdude Is there a proper way to do what I'm attempting to do? — XvaneD, Jun 07 '23 at 08:26
C++ doesn't work like this there is not a 1-1 relation between data in memory and data you'd like to have on disk. Loading data from disk and then typecasting it will NOT results in C++ object having their lifetime started (e.g. new is not called). — Pepijn Kramer, Jun 07 '23 at 08:32
Dynamic structures all must have a fixed part, that tells if the current object is dynamic, and if it is then how many elements are in the dynamic parts.You can read this fixed part into a suitable structure. Then read the dynamic parts separately. Then combine the fixed structure and the dynamic parts into a ***new*** object, using a specific class to hold the run-time data in a much better format (like `std::string` for strings, or some [chrono objects](https://en.cppreference.com/w/cpp/chrono) for dates and times, etc). — Some programmer dude, Jun 07 '23 at 08:33
`std::vector` does not contain its data (it's dynamically allocated). So you cannot do such a cast. You'll need to read your file and parse your values individually. For example, you could add a constructor which accepts a file name. — Fareanor, Jun 07 '23 at 08:33
You probably will need to use a 3rd party serialization library since binary serialization is quite tricky to get right (accross multiple compilers/platforms and compiler settings) — Pepijn Kramer, Jun 07 '23 at 08:33
@PepijnKramer I will be deploying in a container so it doesn't need to be crossplatform but if you have any pointers regarding a serialization library it would be really helpful. — XvaneD, Jun 07 '23 at 08:38
I only have (limited) experiance with protobuf (from using gRPC) since I never had to do complex binary serialization, but I did com across this page a while back [serialization libraries performance](https://github.com/felixguendling/cpp-serialization-benchmark). This will not tell you anything about the learning curve you will run into (maybe others can give some feedback, even though looking for recommendations is considered out of scope for SO, usually opinion based) — Pepijn Kramer, Jun 07 '23 at 08:46
It's only non-'manual' *if and only if* you can guarantee that the C++ `struct` layout in memory exactly matches what you are reading from the file. And there is nothing anywhere that allows you to make that assumption. Especially when `std::vector` is involved. — user207421, Jun 07 '23 at 08:51
@user207421 ah sorry, I misunderstood the first answer compared to my first question. You are right — XvaneD, Jun 07 '23 at 08:54

Parse binary into a structure dynamically

0 Answers0