protobuf-net does not deserialize DateTime.Kind correctly

Question

using protobuf-net.dll Version 1.0.0.280

When I deserialize a DateTime (wrapped in an object), the date/time is ok but the DateTime.Kind property is 'Unspecified'

Consider this test case to serialize/deserialize a DateTime.

[TestMethod]
public void TestDateTimeSerialization()
{
    var obj = new DateTimeWrapper {Date = DateTime.UtcNow};
    obj.Date = DateTime.SpecifyKind(obj.Date, DateTimeKind.Utc);
    var serialized = obj.SerializeProto();
    var deserialized = serialized.DeserializeProto<DateTimeWrapper>();
    Assert.AreEqual(DateTimeKind.Utc, deserialized.Date.Kind);
}

public static byte[] SerializeProto<T>(this T item) where T : class
{
    using (var ms = new MemoryStream())
    {
        Serializer.Serialize(ms, item);
        return ms.ToArray();
    }
}

public static T DeserializeProto<T>(this byte[] raw) where T : class, new()
{
    using (var ms = new MemoryStream(raw))
    {
        return Serializer.Deserialize<T>(ms);
    }
}

The Assert fails, the Kind == Unspecified

Addendum

As a result of protobuf-net not serializing this property (see below) one solution is to assume DateTimeKind is equal to Utc when displaying dates on the client side (only where you know it should be UTC of course):

public static DateTime ToDisplayTime(this DateTime utcDateTime, TimeZoneInfo timezone)
{
    if (utcDateTime.Kind != DateTimeKind.Utc)//may be Unspecified due to serialization
        utcDateTime = DateTime.SpecifyKind(utcDateTime, DateTimeKind.Utc);
    DateTime result = TimeZoneInfo.ConvertTime(utcDateTime, timezone);
    return result;
}

This saves you having to assign to each DateTime property on the receiving side.

Answer 1

protobuf.net has to maintain compatibility with the protobuf binary format, which is designed for the Java date/time datatypes. No Kind field in Java -> No Kind support in the protobuf binary format -> Kind not transferred across the network. Or something along those lines.

As it turns out, protobuf.net encodes the Ticks field (only), you'll find the code in BclHelpers.cs.

But feel free to add another field in your protobuf message definition for this value.

Answer 2

As an extension to Ben's answer... strictly speaking, protobuf has no definition of time, so there is nothing to retain compatibility with. I'm tempted to add support for this in v2, but sadly it would add 2 bytes per value. I have yet to think about whether this is acceptable... for example, I could perhaps default to "unspecified" so that only explicitly local or UTC dates have a value.

Answer 3

Another solution is to change kind property for DTO and always set it to UTC. This may not be acceptable for all the application, but works for me

class DateTimeWrapper 
{
    private DateTime _date;

    public DateTime Date 
    {
        get { return _date; }
        set { _date = new DateTime(value.Ticks, DateTimeKind.Utc);}
    }
}

Update

After using protobuf for more than a year and integrating C#, Java, Python and Scala I came to conclusion that one should use a long representation for the DateTime. For example using UNIX time. It's painful to translate C# DateTime protobuf object to other languages DateTime. However, something as simple as long is understood by all.

Answer 4

It might make more sense for protobuf to automatically deserialize the DateTime with the UtcKind, that way if you are using Utc as your base, which I think is best practice anyway, you wont have any issues.

Answer 5

Here's an implementation for a workaround. Let me know if you can find a better solution. Thanks!

[ProtoContract(SkipConstructor = true)]
public class ProtoDateTime
{
    [ProtoIgnore]
    private DateTime? _val;

    [ProtoIgnore]
    private DateTime Value
    {
        get
        {
            if (_val != null)
            {
                return _val.Value;
            }
            lock (this)
            {
                if (_val != null)
                {
                    return _val.Value;
                }
                _val = new DateTime(DateTimeWithoutKind.Ticks, Kind);
            }
            return _val.Value;
        }
        set
        {
            lock (this)
            {
                _val = value;
                Kind = value.Kind;
                DateTimeWithoutKind = value;
            }
        }
    }

    [ProtoMember(1)]
    private DateTimeKind Kind { get; set; }
    [ProtoMember(2)]
    private DateTime DateTimeWithoutKind { get; set; }


    public static DateTime getValue(ref ProtoDateTime wrapper)
    {
        if (wrapper == null)
        {
            wrapper = new ProtoDateTime();
        }
        return wrapper.Value;
    }

    public static DateTime? getValueNullable(ref ProtoDateTime wrapper)
    {
        if (wrapper == null)
        {
            return null;
        }
        return wrapper.Value;

    }

    public static void setValue(out ProtoDateTime wrapper, DateTime value)
    {
        wrapper = new ProtoDateTime { Value = value };
    }

    public static void setValue(out ProtoDateTime wrapper, DateTime? newVal)
    {
        wrapper = newVal.HasValue ? new ProtoDateTime { Value = newVal.Value } : null;
    }
}

Usage:

[ProtoContract(SkipConstructor = true)]
public class MyClass
{
    [ProtoMember(3)]
    [XmlIgnore]
    private ProtoDateTime _timestampWrapper { get; set; }
    [ProtoIgnore]
    public DateTime Timestamp
    {
        get
        {
            return ProtoDateTime.getValue(ref _timestampWrapper);
        }
        set
        {
            return ProtoDateTime.setValue(out _timestampWrapper, value);
        }
    }

    [ProtoMember(4)]
    [XmlIgnore]
    private ProtoDateTime _nullableTimestampWrapper { get; set; }
    [ProtoIgnore]
    public DateTime? NullableTimestamp
    {
        get
        {
            return ProtoDateTime.getValueNullable(ref _nullableTimestampWrapper);
        }
        set
        {
            return ProtoDateTime.setValue(out _nullableTimestampWrapper, value);
        }
    }

}

Answer 6

Assuming you only need a single DateTimeKind (i.e. UTC or Local) there's a simple (though not pretty) solution.

Since internally protobuf-net converts DateTime's into Unix-Time representation it has a single DateTime value representing the Unix epoch (1970/01/01) to which it adds the relevant delta each time.

If you replace that value using reflection with a UTC or Local DateTime value, all your DateTimes will have that specified DateTimeKind:

typeof (BclHelpers).
    GetField("EpochOrigin", BindingFlags.NonPublic | BindingFlags.Static).
    SetValue(null, new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc));

You can read more about in on my blog

Answer 7

Starting with protobuf-net 2.2 (see commit), there is a possibility to opt-in to the serialization of DateTime.Kind. You can set a global flag. The corresponding issue on github (still open).

And here is a usage example in connection with NServiceBus.

Disclaimer: This won't help for the old protobuf-net version the OP is referring to, but this is an old question, and may be helpful for others.