Google Protocol Buffers - protobuf-net deserialization not working properly

Question

I am experiencing difficulties successfully deserializing a byte array that had been serialized with a pre-compiled protobuff serializer class ModelSerializer. Using this, the serialization and deserialization of classes within my defined data model MyData works when I serialize to a file and deserialize from the file to MyData.

However, I have another requirement, which is to serialize MyData to a byte array, and deserialize the byte array to MyData. Below is the basic class called MyDataConverter that has 2 static methods, one to convert MyData to the byte array and the other to convert a byte array to MyData.

I could serialize MyData to a byte array successfully and write it onto a MemoryStream. However, when I deserialize the byte array, I do get a non-null instance of MyData, but all the data in it is lost (zero default, or null values for custom types).

Could I have gone wrong here? Note that I did use similar code to successfully serialize to a file and deserialize from a file, so it probably isnt something wrong with MyModel nor the pre compiled ModelSerializer. But I could paste some of that info here if necessary.

public class MyDataConverter
{
    public static byte [] MyDataToBytes (MyData myData)
    {
        MemoryStream stream = new MemoryStream();
        ModelSerializer serializer = new ModelSerializer();

        serializer.Serialize (stream, myData);
        byte [] bytes = stream.ToArray();
        Stream.Close();

        return bytes;
    }

    public static MyData BytesToMyData (byte [] bytes)
    {
        MyData myData = null;

        MemoryStream stream = new MemoryStream();
        stream.Write (bytes, 0, bytes.Length);

        ModelSerializer serializer = new ModelSerializer();

        myData = (MyData) serializer.Deserialize (stream, myData, typeof (MyData));
        stream.Close();

        return myData;
    }
}

score 10 · Answer 1 · answered Feb 21 '13 at 16:08

10

This is the problem:

MemoryStream stream = new MemoryStream();
stream.Write (bytes, 0, bytes.Length);

ModelSerializer serializer = new ModelSerializer();
myData = (MyData) serializer.Deserialize (stream, myData, typeof (MyData));

When you try to deserialize, you're still at the end of the stream. You could just seek to the beginning afterwards, but it would be simpler to change the code to pass the byte array to the constructor:

MemoryStream stream = new MemoryStream(bytes, false);        
ModelSerializer serializer = new ModelSerializer();
myData = (MyData) serializer.Deserialize (stream, myData, typeof (MyData));

answered Feb 21 '13 at 16:08

Jon Skeet

1,421,763
867
9,128
9,194

Thank you so much! Works perfectly. Just to improve my understanding, is it because after using stream.Write to write the byte array to the stream, the stream position is at the end, and therefore when i pass the stream to the deserializer, it is not correct? Whereas constructing a new memory stream with the byte array will not change the stream position (or the constructor has already helped move it back). Is my understanding correct? – Nicholas Ang Feb 21 '13 at 16:40
Yes, that's exactly it - the constructor gives it data, but the "cursor" is still positioned at the start of the stream. (Also, using this form you don't end up with another copy of the data - the stream is basically just a view onto the original byte array.) – Jon Skeet Feb 21 '13 at 16:41
Awesome.. thank you so much for such a quick and concise reply! – Nicholas Ang Feb 21 '13 at 16:50
If only we had a penny for every time it was one of: "you haven't rewound the stream", "don't use `GetBuffer()` unless you track the `.Length`", or "don't use `Encoding` to turn non-text binary into a `string`"... – Marc Gravell Feb 21 '13 at 17:53
@MarcGravell: Indeed - the other day I started writing up a blog post about "ways in which encodings can bite you" but aborted it as it required too much introduction. Maybe one day... – Jon Skeet Feb 21 '13 at 17:58
@JonSkeet yeah, I keep meaning to add a "common errors with binary", also covering `Stream` errors. I think I'll bite the bullet and write that later. What else does one do in Redmond, eh? Either that or finish reading chapters 15/16 – Marc Gravell Feb 21 '13 at 18:01
@MarcGravell: Maybe we should do this as a joint post, if possible. I'd like to cover Java at the same time, given that the causes are basically the same. – Jon Skeet Feb 21 '13 at 18:11
@Jon I've written some initial notes on g-docs and shared it with you – Marc Gravell Feb 22 '13 at 00:19

score -1 · Answer 2 · answered May 20 '16 at 19:02

-1

Can we pass a list of object like List as an input to serialiser.

answered May 20 '16 at 19:02

chai

1
5

It would be better to post it as comment. – Ilya May 20 '16 at 19:21
Currently, we are developing a web application which displays highcharts for the automobile related data using C#, .NET and highchart technologies. Can we use Protobuf-net serialize and deserialize methods for web applications? We need the JSON serialized string which has the Report object data which is provided as input to highcharts. The report object is a list of object ReportViewModel, which has many lists of data inside it. Considering this scenario, can we use ProtoBuf-net for our web application. – chai May 25 '16 at 14:45
MemoryStream stream = new MemoryStream(); Serializer.Serialize(stream, report); – chai May 25 '16 at 14:46
I am trying to serialize as mentioned above, but it does seem working correct. This stream is not containing all the report data !!! – chai May 25 '16 at 14:47

Google Protocol Buffers - protobuf-net deserialization not working properly

2 Answers2