-1

I am using Avro and schema registry. I am wondering how to let the consumer read the data by specifying the schema id in Java project without having a Java class for the Avro schema. From my understanding, we could not specify the schema version with a Generic record, and I am wondering how the Generic record handle the schema evolution. For the Specific Record, we must generate the Java class for the Avro schema in the Java project, and it may be not convenient in a cloud application. Is there any way to choose the schema Id as a configuration in my Java project without downloading the schema file from schema registry? Thanks a lot!

Shan
  • 177
  • 3
  • 12
  • Can you clarify "may not be convenient"? You would always need to update your consumer code to read new fields of records, regardless of the serialization format. – OneCricketeer Feb 10 '21 at 23:06
  • I agree with you. I was thinking that if there could be an easier way to just put a schema id configuration into the consumer Java project like putting a topic name config, and then we may not need to generate the schema class. I think I got the answer from you in the other question for generating schema class in the cloud project. Thanks a lot!! – Shan Feb 11 '21 at 03:49

1 Answers1

0

You can look at the source code here that gets the schema ID from the message that there's no way to bypass / override the ID lookup.

That's not to say you could write your own deserializer or a small Kafka Streams job that would override the ID within the messages before forwarding to the Confluent deserializer

Avro requires a reader and writer schema, and the ID lookup is for the writer. When you give a specific record, you're defining the reader which doesn't necessarily have to come from the registry, only needs to be compatible with the written messages

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • Thank you again! So helpful! I looked into the source code, and thanks for detailed and clear explanation! – Shan Feb 11 '21 at 21:22