1

Working on a project using the .net confluent kafka client, getting an exception when deserializing an avro map type. Is there a way of doing this in c#?

The project is producing and consuming message from kafka. There are no issues around consuming anything other than map types.

Avro schema snippet:

  {
    "name": "DurationMargins",
    "type": {
      "type": "map",
      "values": {
        "type": "map",
        "values": "double"
      }
    }
  },

Avrogen creates the following property from this:

private IDictionary<string,IDictionary<string,System.Double>> _DurationMargins;

But when trying to consume the messages out of Kafka, receive the following exception:

Avro.AvroException : Unable to find type IDictionary<string,System.Double> in all loaded assemblies in field DurationMargins
Antonf26
  • 11
  • 3
  • 2
    why cant you de-serialize into a proper model class ? – Kunal Mukherjee Jul 11 '19 at 07:38
  • Post your code. How did you try to deserialize the data? `IDictionary` is an *interface*, not a class that can be instantiated – Panagiotis Kanavos Jul 11 '19 at 07:40
  • Also post the *full* exception text returned by `Exception.ToString()`. This includes the call stack and any inner exceptions which show *exactly* which method raised the exception. Both [Avro's](https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Specific/ObjectCreator.cs) and [Cofluent Kafka's](https://github.com/confluentinc/confluent-kafka-dotnet) .NET source code is available on Github. You can check the method that threw to find out what it was expecting – Panagiotis Kanavos Jul 11 '19 at 07:49
  • And the Avro code doesn't seem to handle *nested maps*. Why are you using nested maps? If you want to store complex objects (JSON or not) you can use records. Check [this SO question](https://stackoverflow.com/questions/28163225/how-to-define-avro-schema-for-complex-json-document) – Panagiotis Kanavos Jul 11 '19 at 07:53
  • @PanagiotisKanavos Yeah seems like the issue was nested maps. I've changed this to a map of records and it's de-serializing succesfully. – Antonf26 Jul 11 '19 at 09:09

1 Answers1

0

This was an issue because we were trying to use nested maps, as @PanagiotisKanavos.

Changing the schema to use a map of record types has fixed my issue.

Thanks for the help

Antonf26
  • 11
  • 3