0

I was running confluent(v5.5.1) s3 sink connector with below config:

"value.converter":"io.confluent.connect.avro.AvroConverter",
"value.converter.schema.registry.url":"http://registryurl",
"value.converter.value.subject.name.strategy":"io.confluent.kafka.serializers.subject.RecordNameStrategy",
......

And got below error in the log like:

DEBUG Sending GET with input null to http://registryurl/schemas/ids/309?fetchMaxId=false
DEBUG Sending POST with input {......} to http://registryurl/MyRecordName?deleted=true
Caused by: org.apache.kafka.common.errors.SerializationException: Error retrieving Avro value schema version for id 309
Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Subject not found.; error code: 40401

There are 2 questions that baffles me here:

  1. Why the sink connector sends additional POST request to schema registry given it's just a consumer? And I have successfully received messages when using a standard kafka consumer, which ONLY sends a GET request to the schema registry.

  2. As per this docs and official doc, the schema subject format will be like SubjectNamingStrategy-value or -key. However judging by the log, it does not suffix the request with "-value". I have tried all the 3 strategies and found ONLY the default TopicNameStrategy works as expected.

Appreciated if anyone could shed some light here.

Thanks a lot

gemitroy
  • 1
  • 1
  • Are you sure that the producer isn't the one sending the POST? And the docs you linked to are for Flink, not written by Confluent – OneCricketeer Feb 08 '21 at 16:10
  • Hi @OneCricketeer. Yes I'm sure, because the log is from my s3 sink connector only. If you are looking for an official Confluent doc, this [link](https://docs.confluent.io/platform/current/schema-registry/serdes-develop/index.html#how-the-naming-strategies-work) should be the one, which also states so. – gemitroy Feb 09 '21 at 01:11
  • If the subject in the registry already exists, the POST is effectively a no-op. Code is open source if you really want to dig into why it happens. Also, maybe it's a version issue, but other people have had issues with the strategies in the past https://stackoverflow.com/questions/53521661/kafka-connect-not-working-with-subject-strategies – OneCricketeer Feb 09 '21 at 14:43

0 Answers0