Serializing/Deserializing Protocol Buffers

Question

I am currently working with Protocol Buffers (version 3 C#). I am sending messages back and forth to different services. and currently trying to save some data stored in certain messages to a database (could be of any kind really).

The problem being that byte[] is created as the type ByteString and List< T> is created as RepeatedField< T>. Now the issue I have with these are that I haven't managed to serialize or deserialize them 100% successfully.

Basic types works like a charm, but Protocol Buffers own types can be a challenge.

I tried AutoMapper and created maps which made ByteString possible to serialize/deserialize but the RepeatedField is hard since it is readonly and have a private setter and is not settable in the constructor.

I cannot manage to get AutoMapper to map correctly to it, I wouldn't be surprised if there are more troubles along the way and was wondering if there are easier ways to save messages in a database?

I have read older versions of protobuf where (if I'm not mistaken) there were so called builders that you could access properties for each message which were mutable and would have made serializing/deserializing much easier. Or is there a more apparent way to access the data and store it in a database that I am just not seeing?

I understand the reason why messages are immutable but is there really no straight-forward way of saving the containg data to a database? Feels like an important feature.

PS: I am aware that there is a protobuf-net solution which handles serializing/deserializing but it only supports protobuf v.2 and I am heavy dependent on v.3 features such as Any.

score 7 · Answer 1 · answered Sep 25 '17 at 18:54

7

I have got AutoMapper 6.1.1 to map to protobufs 3 RepeatedField<>. I'm sure the reflection could be improved, but AutoMapper configuration is:

void Configure(IMapperConfigurationExpression cfg)
{
    cfg.CreateMap<ProtoThings, HasListOfThings>().ReverseMap();

    bool IsToRepeatedField(PropertyMap pm)
    {
        if (pm.DestinationPropertyType.IsConstructedGenericType)
        {
            var destGenericBase = pm.DestinationPropertyType.GetGenericTypeDefinition();
            return destGenericBase == typeof(RepeatedField<>);
        }
        return false;
    }
    cfg.ForAllPropertyMaps(IsToRepeatedField, (propertyMap, opts) => opts.UseDestinationValue());
}

answered Sep 25 '17 at 18:54

Samuel Allan

71
1
2

Great! was a long time since i dabbled with that project, but for curiosity's sake I will definitely check this out. My solution was to download the source code of the protoc client and change the code generator so all repeated fields had public setters if I am not mistaken. Prefer your solution if it works though. I will let you know, thanks. – user4319995 Nov 16 '17 at 14:07
This solution worked for me with proto3 and AutoMapper 6.2.2 – Peter Wishart Jan 15 '18 at 13:22

score 0 · Answer 2 · answered Apr 05 '19 at 16:22

If you know which member in the model is a protobuf repeated field, you can specify to map the destination values, instead of the repeated field itself. This avoids the headache of the protobuf repeated field not having a public 'set' defined for it.

CreateMap<Model, ProtoModel>().ForMember(dest => dest.MyRepeatedCollection, opt => opt.UseDestinationValue());

Serializing/Deserializing Protocol Buffers

2 Answers2

Linked