4

Are there any good existing C++ serialization libraries that support partial serialization?

By "partial serialization" I mean that I might want to save the values of 3 specific members, and later be able to apply that saved copy to a different instance. I'd only update those 3 members and leave the others intact.

This would be useful for synchronizing data over a network. Say I have some object on a client and a server, and when a member changes on the server I want to send the client a message containing the updated value for that member and that member only. I don't want to send a copy of the whole object over the wire.

boost::serialization at a glance looks like it only supports all or nothing.

Edit: 3 years after originally writing this I look back at it and say to myself, 'wut?' boost::serialization lets you define what members you want saved or not, so it would support 'partial serialization' as I seem to have described it. Further, since C++ lacks reflection serialization libraries require you to explicitly specify each member you're saving anyway unless they come with some sort of external tooling to parse the source files or have a separate input file format that is used to generate C++ code (e.g. what Protocol Buffers does). I think I must have been conceptually confused when I wrote this.

Joseph Garvin
  • 20,727
  • 18
  • 94
  • 165

2 Answers2

4

You're clearly not looking for serialization here.

Serialization is about saving an object and then recreating it from the stream of bytes. Think video games saves or the session context for a webserver.

Here what you need is messaging. Google's FlatBuffers is nice for that. Specify a message that will contain every single field as optional, upon reception of the message, update your object with the fields that do exist and leave the others untouched.

The great thing with FlatBuffers is that it handles forward and backward compatibility nicely, as well as text and binary encoding (text being great for debugging and binary being better for pure performance), on top of a zero-cost parsing step.

And you can even decode the messages with another language (say python or ruby) if you save them somewhere and want to throw together a html gui to inspect it!

Matthieu M.
  • 287,565
  • 48
  • 449
  • 722
  • I wouldn't say I'm "clearly not looking for serialization." Perhaps serialization libraries don't usually consider this use case, but I think it's a pretty natural extension of full serialization to wish to be able to save only a subset of the members of an object. Sending data over the network is only one possible use case. – Joseph Garvin Nov 28 '09 at 19:50
  • I don't mean to seem pedantic, but Serialization is about saving enough information that you will be able to rebuild a semantically identical clone from it. A subset of the attributes might be sufficient for that (if you can compute the rest). Here however we have a different case: it's more of an Observer Pattern really. And an Observer Pattern is really about communication... and so we are talking about events (or messages). Whether a network is involved does not really matter (though it complicates things a bit). – Matthieu M. Nov 29 '09 at 12:40
  • I don't mean to seem pedantic either, but if serialization is saving enough information to create a semantically identical clone, partial serialization can be saving enough information to create a partially semantically identical 'clone' ;) Or rather, identical from some (narrower than the original) perspective. – Joseph Garvin Nov 30 '09 at 14:47
  • @MatthieuM. Wikipedia says *"Protocol Buffers are a method of serializing structured data. As such, they are useful in developing programs to communicate with each other over a wire or for storing data."* Consistent with their [definition of serialization](http://en.wikipedia.org/wiki/Serialization), but Boost says *"Here, we use the term 'serialization' to mean the reversible deconstruction of an arbitrary set of C++ data structures to a sequence of bytes."* Perhaps idiomatic C++ concepts of inserters/extractors are more in line with *that* definition of serialization, while there are others. – HostileFork says dont trust SE Nov 14 '12 at 19:11
  • As of 2018, it worths mentioning flatbuffers here: https://google.github.io/flatbuffers/index.html – Dmitrii Volosnykh Feb 14 '18 at 13:39
  • 1
    @DmitryVolosnykh: Indeed; which is why now library recommendations are not accepted as questions => answers tend to degenerate with time. – Matthieu M. Feb 14 '18 at 15:06
  • @MatthieuM. Anyway this Q&A was useful for me. Maybe someone will benefit from mentioning other solutions on serialisation too. – Dmitrii Volosnykh Feb 15 '18 at 20:50
  • @DmitryVolosnykh: They may indeed, which is why closing a question does not delete it. A better format for software recommendations would be a wiki-style page though, as libraries appear and disappear, so the recommendation needs constant updates. Plus there are pros and cons to each that need be documented so the user can make an informed choice based on their own requirements. – Matthieu M. Feb 16 '18 at 07:38
0

Although I'm not familiar with them, you could also check out Google's Protocol Buffers .

Stephen Nutt
  • 3,258
  • 1
  • 21
  • 21