0

Hello I'm trying to connect my app into our Amazon MSK Cluster with this code However it always fails to connect into the Amazon MSK cluster

kafka:
bootstrap-servers: b-1.kafka-cluster.XXX.XXXX.kafka.me-south-1.amazonaws.com:9092
consumer:
  retries: 3
  retry-backoff-ms: 1000
  replication-factor: 1
  reconnect-backoff-ms: 1000
  max-poll-records: 1000
  session-timeout-ms: 1000
  bootstrap-servers: b-1.kafka-cluster.XXX.XXXX.kafka.me-south-1.amazonaws.com:9092
  group-id: userGroups
  auto-offset-reset: earliest
  key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
  value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
producer:
  group-id: userGroups
  bootstrap-servers: b-1.kafka-cluster.XXX.XXXX.kafka.me-south-1.amazonaws.com:9092
  key-serializer: org.apache.kafka.common.serialization.StringSerializer
  value-serializer: org.apache.kafka.common.serialization.StringSerializer

Is it possible to connect to Amazon MSK using this kind of Command because when I run my Kafka Cluster locally it is working the not managed one by amazon msk

mikster21
  • 1
  • 1
  • It definitely is possible. What's the error message? Have you made sure, that you don't have any security group settings in AWS, which would block the access? – dunni Jan 17 '21 at 11:37
  • 2021-01-17 19:39:26.147 WARN 7136 --- [| adminclient-2] org.apache.kafka.clients.NetworkClient : [AdminClient clientId=adminclient-2] Connection to node -1 (b-1.kafka-cluster2.XXX.XX.kafka.me-south-1.amazonaws.com/XXX.XX.XXXX:9092) could not be established. Broker may not be available. This is the error message. I'm running this app in my local and I'm tunneled with bastion host. I had whitelist my public IP on the security groups which the msk have. – mikster21 Jan 17 '21 at 11:42
  • I had faced the same issue and I resolved it by making sure the MSK and SpringBoot App (EC2) are in the same VPC. Also I added Incoming and outgoing rules for Security Groups of both Ec2 and MSK to allow communication either ways. – sumanth.donthula Jan 19 '21 at 09:42
  • @sumanth.donthula may I ask if you are tunneling too from bastion host and you set the bootstrap server into 'localhost:9092' and then set it into your application.properties/yml by the way Are you able to access it from your local machine? – mikster21 Jan 19 '21 at 11:52

0 Answers0