saving data structure c++ to disk

Question

I have a very complex data structure with pointers to various parts of the memory. It takes a while to build and I would like to save the "memory" used by that structure to the disk. Then, when the program is launched again, it would just memory map the file and I could use it. Is there any way to do this ?

Yeah, just walk through your structure and copy the byte array. If its a pointer, just access the data that the pointer points to and copy it. Give an example of the structure and I'll go more indepth. — KrisSodroski, Aug 14 '13 at 14:01
Post a simplified version of the class with, say, two pointers to other stuff and a couple of simple data members, and we'll show you how to write and read it. — Beta, Aug 14 '13 at 14:02
If your object uses pointers you can't just save their values. The next time the program runs there is no guarantee that the objects pointed to will be in the same memory locations. You will need to save everything being pointed as well. — Lachlan Easton, Aug 14 '13 at 14:03
[boost::serialization](http://www.boost.org/doc/libs/1_54_0/libs/serialization/doc/index.html) ? — Paul R, Aug 14 '13 at 14:10
Use a big contiguous buffer, write your own memory allocator that allocates all your data from that buffer, and store offsets from the start of the buffer instead of pointers. Then you can just save and load. The only tricky part is the allocator... ;-) — molbdnilo, Aug 14 '13 at 14:19
It's a tree where nodes have variable amounts of children and may contain vectors of strings. — madreblu, Aug 14 '13 at 14:27
What about using Json or something like that? Then write a routine to walk the tree and form a document, and write the document to serialize and write a routine that reads the Document and initializes a new object to deserialize. This would be comparatively slow, but would be more portable. — gled, Aug 14 '13 at 18:17

score 3 · Answer 1 · answered Aug 14 '13 at 14:12

3

boost::serialization could do it. Note: write the archive version first and then do register_types

answered Aug 14 '13 at 14:12

I tried that, but it says that I need a serialize() method. Is that correct ? – madreblu Aug 14 '13 at 14:28
But how would I write such a method ? I saw examples online but the examples are with really trivial structures not with a complex pointer structure. – madreblu Aug 14 '13 at 15:13

mark · Answer 2 · 2013-08-14T14:18:40.723

It can be done, yes, but you need to be aware of how the structure is actually stored in memory, and how that changes on various platforms (alignment, packing, byte-ordering, primitive type sizes, etc.). Additionally, it would be prudent to change your "pointers to various parts" into "offsets to various parts" since actual memory locations will change between process uses.

example, writing an object like this to disk should be fairly portable:

#pragma pack(1)
typedef struct {
   unsigned char data8[8];
   unsigned char data4[4];
   unsigned char offset[2];
   unsigned char data2[2];
} MyStruct;
#pragma pack() // to return to default

but one like this would be very problematic:

typedef struct _MyStruct {
   unsigned long long data8;
   unsigned int data4;
   struct _MyStruct *nextOne;
   unsigned short data2;
} MyStruct;

score 0 · Answer 3 · answered Aug 15 '13 at 06:50

You could start with Boost.Interprocess and use memory maped files. Your data structure will look a litle differnt (pointers are smart pointers, references forbidden, no virtual functions). But it should be manageable. At least all types you use needs to be constructable in shared memory. Some are (Boost.Container) and some are not (Boost.Any, Boost.Variant).

saving data structure c++ to disk

3 Answers3