protobuf datetimeoffset

Question

Just wanted to share an observation about the R561 version of protobuf-net. When using DateTimeOffSet, an exception

InvalidOperationException (No serializer defined for type: System.DateTimeOffset)

appears:

I added a method with a getProto() and a StreamWriter to write a proto file, and now it works!(and the proto files is totally usable too). But if I comment this method, the same exception occurs. I really don't get it.

Hope this observation could be useful.

---

I will try to be clearer. I have a C# client with some objects using DateTimeOffset parameters. I serialized them with protobuf-net (r561), and added a writeProtoFile() method to write a proto file with the method getProto(). The serialization seems to work perfectly and the proto file is ok too. So because i have my proto file now, i can comment or supress the method writeProtoFile(): i do not need others proto files. So here's my first question:

-> Why the serialization don't work anymore when this method (that is just writting a proto file invoking the getProto() method) is commented or supress? Here's the exception I got:

No serializer defined for type: System.DateTimeOffset.

And when I uncomment the writeProtoFile() comment, it works. Here's the method:

public static void writeProtoFile(String proto)
{
    StreamWriter file = new StreamWriter("c:\\MyprotoFiles\\MyProtoFile.proto");
    file.Write(proto);
    file.Close();

}

I need this object to be consumed by a java client. The java class generated with the proto compiler seems ok, but when I deserialize it, I got an exception :

com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either than the input has been truncated or that an embedded message misreported its own length.

I think, the reason is the DateTimeOffset class generated (In the proto, dateTimeOffset contains nothing)

message DateTimeOffset {
}

The type DateTimeOffset exists in Java, so here's my second question: -> Is there any way that a dateTimeOffset parameter in C# can be serialialized and then, be a dateTimeOffset parameters in java after deserialization?

Answer 1

Is there any way that a dateTimeOffset parameter in C# can be serialialized and then, be a dateTimeOffset parameters in java after deserialization?

There is no defined .proto-based handling of DateTime or DateTimeOffset values for any language, so no there is no even-remotely-guaranteed way of transferring such data between platform via protobuf (or any particular implementation, such as protobuf-net). Additionally, while there is a type called DateTimeOffset on 2 different platforms, that by itself is not enough to guarantee that they have similar semantics / ranges / etc.

For any cross-platform scenarios, I would recommend using just very basic data, perhaps even just something like an integer (64-bit) to store the offset in milliseconds into the 1970 epoch. Or something similar.

Why the serialization don't work anymore when this method (that is just writting a proto file invoking the getProto() method) is commented or supress?

protobuf-net makes zero use of any getProto method or a writeProtoFile method. I would be very cautiously dubious that commenting/uncommenting this is changing some internal behaviour, and would need a concrete repro to investigate. It sounds extremely unlikely, to be honest. Caveat: there is a Serializer.GetProto<T> method, but that does something very different (and is superseded with GetSchema(Type) in the v2 API).

It is, however, entirely correct for it to say:

No serializer defined for type: System.DateTimeOffset.

for the very simple reason that I have not defined a standard serializer for this type. If you can define some standard handling for this, you can probably use SetSurrogate to hook it up to whatever wire representation you choose.

Re:

While parsing a protocol message, the input ended unexpectedly in the middle of a field

That should not happen in any way. That sounds like an unrelated problem, most likely either data corruption when transferring the data (for example, incorrect encoding of the data), or overwriting a pre-existing file without truncating it (leaving garbage at the end). If you can illustrate exactly how you are transferring the binary between the platforms I can probably advise more, however the first thing to investigate here is: is the binary data that you sent the same as the binary data you received (nothing to do with protobuf - just a simple: did you transfer my BLOB correctly? was it the same length? was every byte the same?)

Answer 2

I had the same problem and wrote a wrapper object for proto serialization (see end of answer for code). It has separate fields for Ticks and the Offset. To make work with the helper easier, I have added some operators.

You can assign a DateTimeOffset object directly to the helper class:

[ProtoContract]
public class ExampleContract {
    [ProtoMember(1)]
    public ProtoDateTimeOffset LastLoginTime { get; set; }
}

var contract = new ExampleContract
{
    LastLoginTime = DateTimeOffset.Now
};

When deserializing, you can either work with the ProtoDateTimeOffset instance directly or create an overload in your DTO:

public DateTimeOffset LastLogin { get; set; }

[ProtoMember(4)]
public ProtoDateTimeOffset? ProtoLastLogin
{
    get => LastLogin;
    set => LastLogin = value;
}

This is a C# specific implementation and I don't know what languages you can work with ticks and offsets on but conversion in that languages should be trivial.

---

The actual code of the helper class:

/// <summary>
/// Proto contract that represents a <see cref="DateTimeOffset"/>
/// </summary>
[ProtoContract]
public class ProtoDateTimeOffset
{
    /// <summary>
    /// Utc ticks
    /// </summary>
    [ProtoMember(1)]
    public long Ticks { get; set; }

    /// <summary>
    /// The UTC offset in minutes
    /// </summary>
    [ProtoMember(2)]
    public double OffsetMinutes { get; set; }

    /// <summary>
    /// Operator to cast <see cref="ProtoDateTimeOffset"/> to <see cref="DateTimeOffset"/>
    /// </summary>
    /// <param name="other"></param>
    /// <returns></returns>
    public static implicit operator DateTimeOffset?(ProtoDateTimeOffset? other)
    {
        if (other == null)
            return null;
        return new DateTimeOffset(other.Ticks, TimeSpan.FromMinutes(other.OffsetMinutes));
    }

    /// <summary>
    /// Operator to cast <see cref="ProtoDateTimeOffset"/> to <see cref="DateTimeOffset"/> or default
    /// </summary>
    /// <param name="other"></param>
    /// <returns></returns>
    public static implicit operator DateTimeOffset(ProtoDateTimeOffset? other)
    {
        if (other == null)
            return default;
        return new DateTimeOffset(other.Ticks, TimeSpan.FromMinutes(other.OffsetMinutes));
    }

    /// <summary>
    /// Operator to cast <see cref="DateTimeOffset"/> to <see cref="ProtoDateTimeOffset"/>
    /// </summary>
    /// <param name="other"></param>
    /// <returns></returns>
    public static implicit operator ProtoDateTimeOffset? (DateTimeOffset? other)
    {
        if (other == null)
            return null;
        return new ProtoDateTimeOffset
        {
            OffsetMinutes = other.Value.Offset.TotalMinutes,
            Ticks = other.Value.Ticks
        };
    }
}

Answer 3

I use this method to serialize in C#:

MemoryStream ms = new MemoryStream();
Serializer.Serialize<MyObjectType>(ms, myObject);
byte[] array = ms.ToArray();
ms.Close();

Then i send it to an ActiveMQ topic (pub-sub model) Java receive it in asynchronous way, and deserialize it with the previous method (in my 1st answer). I read the arrays and its seems it is not corrupted, but one is signed and the other's not. In C# i have 10 values upper to 126. And those same values become negatives in Java:

byte #58 =  144 in C# , -112 in Java
byte #67 =  160 in C# , -96 in Java

And after many tests, it seems that those values disappears when i don't serialize the dataTimeOffset parameters.

Answer 4

Does this answer your question? though I am not sure if it is supported in protobuf-net.

syntax = "proto3"
import "google/protobuf/timestamp.proto";

message Meeting {
    google.protobuf.Timestamp time = 1;
}

xref

Answer 5

Here's the java's method for deserializing :

public MyObjectProto.MyObject deserialize(byte[] array) {
    try {
        CodedInputStream stream = CodedInputStream.newInstance(array);
        return MyObjectProto.MyObject.parseFrom(stream);
    } catch (IOException ex) {
        displayLogs("Error while deserializing the content of message : " + ex);
        return null;
    }
}

This deserializing method could corrupt it?