0

We have deployed Confluent Platform 6.0 in our Kubernetes cluster. I have created a Kafka topic "test-topic-1" via Kafka REST api. Now I'm trying to publish a simple AVRO message to this topic.

curl --location --request POST 'https://kafka-rest-master.k8s.hip.com.au/topics/test-topic-1' \
--header 'Content-Type: application/vnd.kafka.avro.v2+json' \
--header 'Accept: application/vnd.kafka.v2+json' \
--data-raw '{"value_schema":{"type":"record","name":"User","fields":[{"name":"name","type":"string"}]},"records":[{"value":{"name":"testUser"}}]}'

I get a 500 error response for this request,

{"error_code":500,"message":"Internal Server Error"}

When I check the logs of the kafka rest pod, I can see the following error,

ERROR Request Failed with exception (io.confluent.rest.exceptions.DebuggableExceptionMapper) com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of java.lang.String out of START_OBJECT token at [Source: (org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 17] (through ref erence chain: io.confluent.kafkarest.entities.v2.SchemaTopicProduceRequest["value_schema"]) at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1445)

Am I following the correct steps to publish an AVRO message to a newly created Kafka topic? If so what could be the problem here?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Vidura Mudalige
  • 810
  • 2
  • 18
  • 31
  • 1
    I think value_schema is a a String representation of the schema, so it should be between " and with the various escapes. Example (https://stackoverflow.com/questions/54591443/posting-avro-serialised-data-using-kafka-rest-proxy) – AndD Feb 08 '21 at 15:35

1 Answers1

1

your AVRO schema definition is wrong. It should be defined as the following

{
  "type": "record",
  "name": "recordName",
  "namespace": "namespace",
  "doc": "description",
  "fields": [
    {
      "name": "key",
      "type": {
        "type": "string",
        "avro.java.string": "String"
      }
    }
  ]
}

OR

"fields": [
  {
    "name": "fiedName",
    "type": "string"
  },
]
s0xzwasd
  • 2,895
  • 3
  • 17
  • 26
ChristDist
  • 572
  • 2
  • 8