3

I am trying to connect to a aws msk cluster from spring boot kafka app.

i have aws msk arn ,boot strap server config available.

How to integrate in the application? Any examples

Can i run it from local,i mean connect to AWS MSK from local app instance

msk cluster

Community
  • 1
  • 1
sc cc
  • 75
  • 2
  • 7

2 Answers2

1

Connection to AWS MSK is like connecting to any Kafka cluster.

It depends on how you had setup your MSK ? (VPC - Subnet) Check whether you can access from your local or need to be on VPN.

Also see how your brokers endpoints are configured and exposed - Ex:

  • Plaintext (broker:9092)
  • TLS (broker:9094)

If its plaintext, its as simple as setting bootstrap.servers.

Otherwise, you need to add below mentioned properties to access your MSK cluster.

security.protocol: SSL
ssl.truststore.location: jks
ssl.truststore.password: pwd

Please let me know if you still have questions.

Surendra S
  • 91
  • 1
  • 6
  • We do not have client ssl authentication ON. So do we need the password and location? updated image – sc cc Jun 19 '20 at 13:36
  • Nope, then you dont need those two properties and security.protocol would be PLAINTEXT. – Surendra S Jun 19 '20 at 15:57
  • finally we had to put ssl. asper https://docs.confluent.io/2.0.0/kafka/ssl.html#configuring-kafka-clientsonly along with //props.put("ssl.endpoint.identification.algorithm", "");However did not need trust store !!as probably both app deployed on ec2 and msk are in same vpn? or is this something automacially by aws – sc cc Jun 19 '20 at 20:08
0

If you can connect from kafka-console-* commands (or simpler, kafkacat) then you should be able to connect from any other application.

I suggest you debug using those tools before running anything else.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245