Can a C# model be serialized as an AVRO JSON Schema?

Question

I have found some code here https://learn.microsoft.com/en-us/azure/hdinsight/hdinsight-dotnet-avro-serialization#Scenario2 that does the reverse of what I need:

//Define the schema in JSON
const string Schema = @"{
    ""type"":""record"",
    ""name"":""Microsoft.Hadoop.Avro.Specifications.SensorData"",
    ""fields"":
        [
            {
                ""name"":""Location"",
                ""type"":
                    {
                        ""type"":""record"",
                        ""name"":""Microsoft.Hadoop.Avro.Specifications.Location"",
                        ""fields"":
                            [
                                { ""name"":""Floor"", ""type"":""int"" },
                                { ""name"":""Room"", ""type"":""int"" }
                            ]
                    }
            },
            { ""name"":""Value"", ""type"":""bytes"" }
        ]
}";

//Create a generic serializer based on the schema
var serializer = AvroSerializer.CreateGeneric(Schema);

I would like to take a model that I have created:

[DataContract(Name = "Demo", Namespace = "pubsub.demo")]
public class Demo
{
    [DataMember(Name = "value")]
    public long Value { get; set; }
}

...and serialize this C# model into a JSON AVRO Schema string.

Reason:

I only want to maintain C# models and automatically register these models with Confluent's Schema Registry. To register with schema registry the schema needs to be in a JSON AVRO format (Just like Schema above).

I would prefer not to have both the JSON defined and the C# model. If I had to maintian one, I would prefer to have a C# model.

Late comment: The Confluent-dotnet-kafka library now includes avrogen, a tool to take a schema to a C# object — OneCricketeer, Sep 30 '18 at 06:55
@cricket_007 is there any difference for the consumer if you publish message as `ISpecificRecord` or `GenericRecord`? Or those are still same byte arrays in message? — OlegI, Sep 30 '19 at 13:48
@cricket_007 thanks for the response! What is the prefered way to produce messages if your topic contains different event models? Should I produce **specific** records or **generic** records? Also, as a consumer, I think there is only one way to consume from that topic - is GenericRecord? Because basically you cannot pass an array of types you expect when you build your consumer. Really no samples for these scenarios on the web :( — OlegI, Sep 30 '19 at 13:59
@OlegI You can watch this issue about support for multiple types in C# https://github.com/confluentinc/confluent-kafka-dotnet/issues/746 — OneCricketeer, Sep 30 '19 at 14:44

score 6 · Accepted Answer · answered Jul 26 '17 at 23:37

6

I found what I was looking for in Microsoft.Hadoop.Avro.AvroSerializer.

AvroSerializer.Create<Demo>().WriterSchema.ToString();
// > {"type":"record","name":"pubsub.demo.Demo","fields"[{"name":"value","type":"long"}]}

answered Jul 26 '17 at 23:37

joelnet

13,621
5
35
49

2

It doesn't seem like MS is supporting this library anymore https://github.com/Azure/azure-sdk-for-net/issues/2321#issuecomment-505963259 nor have the nuget packages been updated in several years https://www.nuget.org/packages?q=Microsoft.Hadoop.Avro – Chris Smith Mar 19 '21 at 20:00

score 3 · Answer 2 · answered Apr 15 '20 at 06:20

3

The solution also could be:

string schema = AvroConvert.GenerateSchema(typeof(Demo));

From https://github.com/AdrianStrugala/AvroConvert

answered Apr 15 '20 at 06:20

Adrian

96
5

score 1 · Answer 3 · answered Mar 10 '21 at 18:28

Chr.Avro provides a CLI that lets you go both ways- C# types to Avro schema, and Avro schema to C# types.

From the docs:

dotnet avro create --type ExampleNamespace.ExampleLibrary.ExampleClass --assembly bin/Debug/netstandard2.0/ExampleNamespace.ExampleLibrary.dll

https://engineering.chrobinson.com/dotnet-avro/guides/cli-create

Can a C# model be serialized as an AVRO JSON Schema?

3 Answers3

Linked