protobuf-net documentation or alternatives

Question

Protobuf-net seems to be the fastest and for high-performence needs most recommended serialization library for .NET. I really want to use it, as I need to send hundreds of thousands of objects over the wire.

However, I'm having trouble getting started. The documentation (wiki at github) is quite sparse, especially for v2.

Somehow, you guys out there seem to be able to get going with the lib. How? By reading the sources? Trial and error? Or are there some API-docs / tutorials I'm not aware of? (I'm only aware of the GitHib page.)

Thanks and cheers,

Jan

P.S.: I need to get going using the RuntimeTypeModel (POCO's without attributes).

Do you have an example model that you would like to serialise, for illustration? — Marc Gravell, Sep 15 '13 at 19:55
*plug* heres an article I wrote that can help you get started (incluing v2 usage) http://wallaceturner.com/serialization-with-protobuf-net — wal, Sep 16 '13 at 14:52

konrad.kruczynski · Answer 1 · 2013-09-20T09:41:33.803

Since you've also asked about alternatives...

No need for attribute decoration was one of the reasons behind creating Migrant, fast serialization library with simple API. The library has some ideas that are also present in protobuf (so we are more or less on par in terms of speed and size), but at the same time tries to solve different problems. Among features distinctive from protobuf, there is a difference between empty and null collection and whole serialization is reference based for references and value based for values (well, you can also treat reference as a special kind of value). README at github should be able to answer most of your questions; whether some more detailed info is needed, just ask.

Simple scenario of custom object serialization:

var stream = new MyCustomStream();
var myComplexObject = new MyComplexType(complexParameters);
var serializer = new Serializer();

serializer.Serialize(myComplexObject, stream);

stream.Seek(0, SeekOrigin.Begin);

var myDeserializedObject = serializer.Deserialize<MyComplexType>(stream);

Note that expected type in Deserialize is only used to have nice compile-time type of the deserialized object, you can use general object as well.

Disclaimer: I'm one of the developers.

score 2 · Accepted Answer · answered Sep 15 '13 at 20:28

In protobuf, each member of a type needs an identifying number, because protobuf is number-based (it doesn't send names). As such, the trick is simply: tell it what numbers to use. For example:

class Customer {
    public int Id {get;set;}
    public string Name {get;set;}
}

The simplest way to specify a contract for this would be:

RuntimeTypeModel.Default.Add(typeof(Customer), false).Add("Id", "Name");

which would associate Id with 1 and Name with 2. When using attributes, there is some inbuilt "figure it out yourself" code, which I should really expose on the non-attribute API - things like:

serialize all public fields + properties, in alphabetical order
serialize all fields (public or non-public), in alphabetical order

However: both of these are pretty simple to do with reflection. Note that in either event, using reflection is a good way to run into problems if the types might change at some point.

Additional features that may help, and which I can provide more information on:

a type factory can be specified (globally or per-instance), which can be useful for pre-populating values or using a pool of available objects
surrogate types can be written for complex scenarios - this is useful when most of the model works fine, but one type is just too esoteric to be a good fit for serialization - but an alternative layout can be devised for which you can write conversion code in both directions
many things that looks like a "tuple" will be handles by default - in particular, if it is publicly immutable, and has a constructor that accepts parameters that match all the public members - it will assume to serialize the members in the order specified by the constructor

RuntimeTypeModel.Compile() takes quite some time ... about 3 sec for one added Type with 20 properties. Am I missing something? Cheers, Jan — Knack, Sep 19 '13 at 03:59
@Knack I *suspect* a lot of that is fusion / JIT etc - I have a model in my test rig with 20 properties 3ms (not s) per `Compile()`; and for an intentionally complex model (801 types, 1600 properties) 600ms per `Compile`. I would be interested in seeing what is going on there. Note also: in most cases you do not *need* to call `Compile()` - it will perform per-type compilation automatically as-needed — Marc Gravell, Sep 19 '13 at 08:42
Something was not right with my setup (not sure if I had IntelliTrace enabled doint these runs ...). Anyway: Now I get consistently the expected performance figures - with and without Compile(). Really, really awesome and impressive. Thanks! — Knack, Sep 23 '13 at 06:46
@Knack heh; I dod a lot of performance work - the first thing I always turn off is intellitrace. Also; you need to run in "Release" mode, and **outside** of the IDE - the get meaningful numbers. Otherwise, you're actually just profiling the IDE / debugger. — Marc Gravell, Sep 23 '13 at 07:02

score 0 · Answer 3 · answered Feb 23 '16 at 23:32

Another option is Dasher (available via NuGet).

It's a straightforward, fast and lightweight serialiser and deserialiser for .NET that works on C# types without needing any attributes or other decorations.

var stream = new MemoryStream();

// serialise to stream
new Serialiser<Holiday>().Serialise(stream, christmas);

stream.Position = 0;

// deserialise from stream
var christmas = new Deserialiser<Holiday>().Deserialise(stream);

It uses reflection emit to produce highly optimised serialisation and deserialisation functions at runtime. The underlying encoding is MsgPack which is self describing, so slightly larger than protobuf on the wire, but it does mean you can decode any message you receive including property names, types and values.

Disclaimer: I wrote the library.

protobuf-net documentation or alternatives

3 Answers3