2

I am using a library, and the library requires me to provide the offset of the desired data in the file, so it can use mmap to read the data (I can't edit the code of this library but only provide the offset).

So I want to use flatbuffers to serialize my whole data because there isn't any packing and unpacking in flatbuffers, (I think) which means that it is easy to get the offset of the desired part in the binary file.

But I don't know how to get the offset. I have tried loading the binary file and calculate the offset of the pointer of the desired field, for example, the address of the root is 1111, the address of the desired field is 1222, so the offset of the field in the binary file is 1222 - 1111 = 111 (because there is no unpacking step). But in fact, the offset of the pointer is a huge negative number.

Could someone help me with this problem? Thanks in advance!

daquexian
  • 169
  • 3
  • 17

1 Answers1

2

FlatBuffers is indeed very suitable for mmap. There are no offsets to be computed, since the generated code does that all for you. You should simply mmap the whole FlatBuffers file, and then use the field accessors as normal, starting from auto root = GetRoot<MyRootType>(my_mmapped_buffer). If you want to get a direct pointer to the data in a larger field such as a string or a vector, again simply use the provided API: root->my_string_field()->c_str() for example (which will point to inside your mmapped buffer).

Aardappel
  • 5,559
  • 1
  • 19
  • 22
  • Thanks for your answer! But In my opinion, the library I am using only mmap the desired field by using the offset of the field, but not the whole binary file, so in fact, I need the offset of the field in the binary file. Is there any way to get it? Thanks again. – daquexian Aug 14 '18 at 14:54
  • The only to get an "offset" (rather, a pointer) to a field is by using the API, and thus by having the FlatBuffer already in memory. Not sure what you're trying to do though, mapping the entire FlatBuffer should be just as efficient as mapping one field, since any unused parts of the FlatBuffer won't get paged in anyway. – Aardappel Aug 14 '18 at 18:09
  • Thanks! I found that I misread the library api, the way is just like you have said. Thanks a lot – daquexian Aug 14 '18 at 23:58