Exception while processing schema registry deserialiser kafka

Question

I'm following the confluent link to post kafka messages on changes to mysql table. When I try to consume this message from a springboot application, I'm getting the below exception. How can I fix this so that I can read the message. Sometimes I am able to read the message but I get a serialised version if like consumed message is key: null value : 2foo��䶰Z��䶰Z

Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to com.spring.kafkaexample.springbootkafkaconsumer.model.Foobar
    at com.spring.kafkaexample.springbootkafkaconsumer.listener.KafkaConsumer.consume(KafkaConsumer.java:22) ~[classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_131]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_131]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131]
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:181) ~[spring-messaging-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:114) ~[spring-messaging-5.0.8.RELEASE.jar:5.0.8.RELEASE]

Below is my code

@KafkaListener(topics = "mysql-foobar", groupId = "group_id")
public void consume(ConsumerRecord<String, Foobar> message) {

    System.out.println("consumed message is key: "+message.key() + " value :  "+message.value());
}

Deserializer

@Bean
public Map<String, Object> consumerConfigs() {
    Map<String, Object> props = new HashMap<>();
    props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:9092");
    props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, KafkaAvroDeserializer.class);
    props.put(ConsumerConfig.GROUP_ID_CONFIG, "avro");

    return props;
}

score 0 · Answer 1 · answered Mar 16 '19 at 20:14

If you are getting String cannot be cast to com.spring.kafkaexample.springbootkafkaconsumer.model.Foobar, then that means you have a producer sending a string into your topic, which is not a valid to be made a Foobar instance.

If you are getting the 2foo��䶰Z��䶰Z stuff, then that would mean you're printing out the UTF-8 raw bytes of some binary data, such as Avro, and you would be better off extracting the fields of the object rather than relying solely on the implicit toString method.

Exception while processing schema registry deserialiser kafka

1 Answers1