How to compare performance of messagepack-cli and json.net deserializers?

Question

I am trying to compare the performance of two different deserialisation methods in Unity3d, which is based on MonoDevelop's implementation of C# / .NET

Method A) Using MsgPack-CLI

Method B) Using NewtonSoft's Json.NET

Based on this blog post, I was under the impression that Messagepack would be faster for both reading and writing. However, I found that while the write performance is significantly better, the read performance is significantly slower.

For the purpose of testing deserialisation performance, am I doing something wrong in this code?

public void test_msgpack(int _num, Test_Class _obj)
{
    var serializer = MessagePackSerializer.Get< Test_Class >();
    var stream = new MemoryStream();

    serializer.Pack(stream, _obj);

    Stopwatch stopWatch = new Stopwatch ();
    stopWatch.Start ();
    while (_num > 0) {
        _num -= 1;
        stream.Position = 0;
        Test_Class deserializedObject = serializer.Unpack( stream );
    }

    stopWatch.Stop ();
    TimeSpan ts = stopWatch.Elapsed;
    string elapsedTime = String.Format ("{0:00}:{1:00}:{2:00}.{3:00}",
        ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10);

    print ("msgpack read: " + elapsedTime);
}

public void test_json(int _num, Test_Class _obj)
{
    var serializer = new JsonSerializer();
    var stream = new MemoryStream();

    var sw = new StreamWriter(stream);
    var JsonTextWriter = new JsonTextWriter(sw);
    var sr = new StreamReader(stream);
    var JsonTextReader = new JsonTextReader(sr);

    serializer.Serialize(JsonTextWriter, _obj);

    Stopwatch stopWatch = new Stopwatch ();
    stopWatch.Start ();
    while (_num > 0) {
        _num -= 1;
        stream.Position = 0;
        Test_Class deserializedObject = serializer.Deserialize<Test_Class>(JsonTextReader);
    }
    stopWatch.Stop ();
    TimeSpan ts = stopWatch.Elapsed;
    string elapsedTime = String.Format ("{0:00}:{1:00}:{2:00}.{3:00}",
        ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10);

    print ("json read: " + elapsedTime);
}

I am finding the JSON parsing performance to be roughly 100 times faster over 100000 iterations... which seems odd? Here is the form of the class that I am trying to serialise / deserialise:

public class Test_Class
{
    public string i { get; set; }
    public List<float> a { get; set; }
    public List<float> b { get; set; }
    public List<float> c { get; set; }
}

MessagePack is going to serialize/deserialize slower because of the additional complexity, however, the result is a smaller file. If a comparison is easier to imagine: take a 1gb file and stick it in a .zip and send it over the network. It's always going to be faster to compress and send the file rather than sending the 1gb file directly. Rather than thinking which is immediately faster, consider how much network traffic you are doing. Especially consider the network traffic you may have to mobile phones as the network is usually slow. — Kody, Jan 02 '18 at 19:52

score 1 · Accepted Answer · answered Aug 27 '15 at 11:06

After some trial and error I found that the JSON.net code was returning a null object, thus the significanlty faster performance. I remedied this by modifying the test_json method as follows:

public void test_json(int _num, Test_Class _obj)
{
    JsonSerializer serializer = new JsonSerializer();

    MemoryStream stream = new MemoryStream();

    StreamWriter writer = new StreamWriter(stream);
    JsonTextWriter jsonWriter = new JsonTextWriter(writer);
    serializer.Serialize(jsonWriter, _obj);
    jsonWriter.Flush();

    Stopwatch stopWatch = new Stopwatch ();
    stopWatch.Start ();
    while (_num > 0) {
        _num -= 1;
        stream.Position = 0;

        StreamReader reader = new StreamReader(stream);
        JsonTextReader jsonReader = new JsonTextReader(reader);
        Test_Class deserialised_object = serializer.Deserialize<Test_Class>(jsonReader);
    }
    stopWatch.Stop ();
    TimeSpan ts = stopWatch.Elapsed;
    string elapsedTime = String.Format ("{0:00}:{1:00}:{2:00}.{3:00}",
        ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10);

    print ("json read: " + elapsedTime);
}

The jsonWriter.Flush() method is necessary, as is creating a new JsonTextReader on each read.

The results still show JSON.net as being faster with the deserialisation, however, this difference is now smaller. The MsgPack-CLI performance can apparently be improved by using a SerializerGenerator class, though I have yet to implement.

How to compare performance of messagepack-cli and json.net deserializers?

1 Answers1