4

I am having difficulties connecting to my MSK cluster from my EKS cluster even though both clusters share the same VPC and the same subnets.

The security group used by the MSK cluster has the following inbound rules

type protocol port range source
all traffic all all custom SG_ID
all traffic all all anywhere ipv4 0.0.0.0/0

Where SG_ID is the EKS' Cluster security group. The one labeled: EKS created security group applied...

In the EKS cluster, I am using the following commands to test connectivity:

kubectl  run kafka-consumer \
-ti \
--image=quay.io/strimzi/kafka:latest-kafka-2.8.1 \
--rm=true \
--restart=Never \
-- bin/kafka-topics.sh --create --topic test  --bootstrap-server b-1.test.z35y0w.c4.kafka.us-east-1.amazonaws.com:9092 --replication-factor 2 --partitions 1 --if-not-exists

With the following result

Error while executing topic command : Call(callName=createTopics, deadlineMs=1635906680860, tries=1, nextAllowedTryMs=1635906680961) timed out at 1635906680861 after 1 attempt(s) [2021-11-03 02:31:20,865] ERROR org.apache.kafka.common.errors.TimeoutException: Call(callName=createTopics, deadlineMs=1635906680860, tries=1, nextAllowedTryMs=1635906680961) timed out at 1635906680861 after 1 attempt(s) Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: createTopics (kafka.admin.TopicCommand$) pod "kafka-consumer" deleted pod default/kafka-consumer terminated (Error)

Sadly, the second bootstrap server displayed on the MSK Page gives the same result.

nc eventually times out

kubectl  run busybox -ti --image=busybox --rm=true --restart=Never -- nc b-2.test.z35y0w.c4.kafka.us-east-1.amazonaws.com

nslookup fails as well

kubectl  run busybox -ti --image=busybox --rm=true --restart=Never -- nslookup b-2.test.z35y0w.c4.kafka.us-east-1.amazonaws.com
If you don't see a command prompt, try pressing enter.
*** Can't find b-2.test.z35y0w.c4.kafka.us-east-1.amazonaws.com: No answer

Could anyone please give me a hint? Thanks

Felipe Lujan
  • 61
  • 3

1 Answers1

0

I need to connect MSK from my EKS pod. So I searched this doc, I want to share my solution, hope can help others: This my config file:

root@kain:~/work# cat kafkaconfig
security.protocol=SASL_SSL
sasl.mechanism=AWS_MSK_IAM
sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;

sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler

This is my command:

./kafka-topics.sh --list --bootstrap-server <My MSK bootstrap server>:9098 --command-config ./kafkaconfig

For this command, there are 2 preconditions we need to make sure,

  1. one is you have access to aws msk, (I access MSK from my eks pod, and my eks pod has OIDC to access the AWS).
  2. Second is we need to has AWS auth jar file: aws-msk-iam-auth.jar address: https://github.com/aws/aws-msk-iam-auth/releases put it to kafkaclient libs directory or export CLASSPATH=/aws-msk-iam-auth-1.1.4-all.jar

reference doc: https://aws.amazon.com/blogs/big-data/securing-apache-kafka-is-easy-and-familiar-with-iam-access-control-for-amazon-msk/

kain
  • 214
  • 2
  • 4