Connection to MSK using spring kafka

Question

Hi I am using aws MSK and trying to connecting to it using spring kafka client. While sending message to kafka cluster from my code, I get following error-

2020-01-27 21:49:17.141 ERROR 4176 --- [nio-9000-exec-4] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='key' and payload='payload' to topic topic1:

org.apache.kafka.common.errors.TimeoutException: Topic topic1 not present in metadata after 60000 ms.

Here is my config-

spring:
  kafka:
    consumer:
      bootstrap-servers: {server}
      group-id: group_id
      auto-offset-reset: earliest
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.springframework.kafka.support.serializer.JsonDeserializer
      ssl:
        trust-store-location: /tmp/kafka.client.truststore.jks
        key-store-location: /tmp/kafka.client.truststore.jks
        protocol: ssl
    producer:
      bootstrap-servers: {server}
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
      ssl:
        trust-store-location: /tmp/kafka.client.truststore.jks
        key-store-location: /tmp/kafka.client.truststore.jks
        protocol: ssl

I am able to connect to the kafka cluster(produce and consume messages) using command line by following the steps from here- https://docs.aws.amazon.com/msk/latest/developerguide/produce-consume.html

The topic - topic1 does exist, and I can list it from command line

Please help.

Are you able to use the same configs from `console-consumer` or `console-producer`? — OneCricketeer, Jan 28 '20 at 01:29
Also, try focusing on one client first. Your app doesn't need to produce and consume in the same app. (And if it did, you probably want Kafka Streams w/ Spring instead) — OneCricketeer, Jan 28 '20 at 02:13
Yes I am able to use same from console-consumer or console-producer. I have both in same just for testing application — Aashish Katta, Jan 28 '20 at 05:08
Sure, but which one is actually failing? Producer or consumer? Please create a [mcve] — OneCricketeer, Jan 28 '20 at 09:00

Connection to MSK using spring kafka

0 Answers0