0

We are in development and trying to delete the schema for a topic , since the change in incompatible with older schema.

We deleted the schema / subject and tried creating the new schema under the same subject name and schema was successfully created.

However, when we run the application, it is still pointing to same schema ID.

Older schema ID ( for subject 'topic1") : 51

New schema ID ( for subject 'topic1') : 52

Application fails with an error deserializing the message 

    at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:736)
Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 51
Caused by: org.apache.avro.AvroTypeException: Found string, expecting union
    at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:292)
    at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
    at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267)
    at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumRea

root@bas:/# curl -k https://schemaregistry:443/subjects/topic1/versions/latest

{"subject":"topic1","version":15,"id":52,"schema":"{\"type\":\"record\",\"name\":\"TopicOutputOutput\",\"namespace\":\"com.sagar.avsc\",\"fields\":[{\"name\":\"id\",\"type\":\"long\"},{\"name\":\"client\",\"type\":{\"type\":\"record\",\"name\":\"ClientObject\",\"fields\":[{\"name\":\"name\",\"type\":[\"null\",\"string\"]},{\"name\":\"client_id\",\"type\":[\"null\",\"long\"]},{\"name\":\"source_system_id\",\"type\":[\"null\",\"long\"]}]}},  ...

Change: {\"name\":\"client_id\",\"type\":[\"null\",\"long\"]}

You can see that the schema ID is 52 but application is trying older schema ID (51), which is incompatible with the data we are sending

root@bas:/#  curl -k https://schemaregistry:443/schemas/ids/52
{"schema":"{\"type\":\"record\",\"name\":\"TopicOutputOutput\",\"namespace\":\"com.sagar.avsc\",\"fields\":[{\"name\":\"id\",\"type\":\"long\"},{\"name\":\"client\",\"type\":{\"type\":\"record\",\"name\":\"ClientObject\",\"fields\":[{\"name\":\"name\",\"type\":[\"null\",\"string\"]},{\"name\":\"client_id\",\"type\":[\"null\",\"long\"]},{\"name\":\"source_system_id\",

Change: {\"name\":\"name\",\"type\":[\"null\",\"string\"]}

why is application still referring to older schema ID ? And is there a way to delete Schema ID ?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
SunilS
  • 2,030
  • 5
  • 34
  • 62

1 Answers1

0

You can issue a DELETE to /subjects/:name/versions/:version

However, schema ID 51 is still in the topic, and there is no way to delete it from there until those particular messages expire, or are tombstoned in the case of compacted topics.

And if the schemas are not compatible, then I would check the configs of the registry because by default, it doesn't allow backwards incompatible changes.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245