Don't break backwards-compatibility messages in a DDS system

Question

I'm involved in a project which uses DDS as a protocol to communicate and C++ as language. As you know exchanged messages are called Topics. Well, sometimes a team must change a topic definition, as a consequence of this, other software which depends on this topic stops working, it is necessary to update the topic everywhere and recompile. So, my question is, do you know how not to break backwards compatiblity? I have been searching and I have found google protocol buffer, they say this:

"You can add new fields to your message formats without breaking backwards-compatibility; old binaries simply ignore the new field when parsing. So if you have a communications protocol that uses protocol buffers as its data format, you can extend your protocol without having to worry about breaking existing code."

Any other idea?

Thanks in advance.

Answer 1

The OMG specification Extensible and Dynamic Topic Types for DDS (or DDS-XTypes for short) addresses your problem. Quoted from that specification:

The Type System supports type evolution so that it is possible to “evolve the type” as described above and retain interoperability between components that use different versions of the type

Not all DDS implementations currently support XTypes, so you might have to resort to a different solution. For example, you could include a version numbering scheme in your Topic name to avoid typing conflicts between different components. In order to make sure every component receives the right data it needs, you can create a service that has the responsibility to forward between different versions of the Topics as needed. This service has to be aware of the different versions of the Topics and should take care of filling default values and/or transform between their different types. Whether or not this is a viable solution depends, among other things, on your system requirements.

The use of a different type system like Protocol Buffers inside DDS is not recommended if it is just for the purpose of solving the type evolution problem. You would essentially be transporting your PB messages as data opaque to the DDS middleware. This means that you would also lose some of the nice tooling features like dynamic discovery and display of types, because the PB messages are not understood by the DDS middleware. Also, your applications would become more complex because they would be responsible for invoking the right PB de/serialization methods. It is easier to let DDS take care of all that.

Whichever route you take, it is recommended that you manage your data model evolution tightly. If you let just anybody add or remove some attributes to their liking, the situation will become difficult to maintain quickly.

Answer 2

The level of support for protocol buffers with DDS depends on how its implemented. With PrismTech's Vortex for instance, you don't lose content-awareness, nor dynamic discovery nor display of types as both the middleware and its tools DO understand the PB messages. W.r.t. 'populating' your PB-based topic, thats conform the PB-standard and generated transparently by the protoc compiler, so one could state that if you're familiar with protobuf (and perhaps not with OMG's IDL alternative), then you can really benefit from a proper integration of DDS and GPB that retains the global-dataspace advantages yet i.c.w. a well-known/popular type-system (GPB)

Answer 3

You could try wrapping the DDS layer in your own communication layer. Define a set of DDS Types and DDS Profiles that covers all your needs. Then each topic will be define as one of those types associated to one of those profiles.

For example you could have a type for strings, and a binary type. If you use the same compiler for all your software, you can even safely memcopy a C struct to the binary type.

module atsevents {
valuetype DataType {
    public string<128> mDataId;
    public boolean mCustomField1;
    public boolean mCustomField2;
};
valuetype StringDataType : DataType {
    public string<128> mDataKey; //@key
    public string<1024> mData;
};
valuetype BinaryDataType : DataType {
    public string<128> mDataKey; //@key
    public sequence<octet, 1024> mData;
    public unsigned long mTypeHash;
}
}