1

I try to connect my aws msk which is in private vpc with msk connector to my cloud.elastic.io elasticsearch. MSK cluster and elastic.io are in same region us-west-2

I use https://www.confluent.io/hub/confluentinc/kafka-connect-elasticsearch but always get error

The connector configuration is invalid. Message: Connector configuration is invalid and contains the following 4 error(s): Could not connect to Elasticsearch. Error message: java.util.concurrent.ExecutionException: java.net.ConnectException

connection.url/password/username are correct I checked them multiple times.

Also I'm able to do connection from https://confluent.io/ but not from aws msk connector.

I tried to use default vpc and create a msk cluster but it also didn't work.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Vahan Hovsepyan
  • 11
  • 1
  • Please dont add the tag back, Elastic connector works with any Kafka cluster, not only Confluent Platform, and there is no connection to confluent.io after you download the connector into the worker – OneCricketeer Aug 09 '23 at 16:55
  • @OneCricketeer https://github.com/confluentinc/kafka-connect-elasticsearch, so this repo owned by Confluent Platform, and I have error with the java connection, doesn't that mean it's related to Confluent Platform ? Or I'm missing something ? – Vahan Hovsepyan Aug 10 '23 at 18:48
  • Confluent, the company, publishes a plugin to be used with Apache Kafka. It is not related to the software stack called "Confluent Platform". Similarly, Aiven has an [OpenSearch Kafka Connector](https://github.com/Aiven-Open/opensearch-connector-for-apache-kafka) that may also work with Elasticsearch to be used with Kafka Connect, which is Apache licensed, not specific to Confluent. – OneCricketeer Aug 10 '23 at 19:42
  • @OneCricketeer ok, thank you for information – Vahan Hovsepyan Aug 10 '23 at 21:06
  • In any case, it'd be great if you could show the complete config and stacktrace rather than only one line of it. Also note that the Elasticsearch sink connector is version specific and may not work with Elasticsearch versions higher than 6.x or 7.x last I checked. You may also need to allow the AWS VPC to access Elastic cloud; it's not enough they are in the same region. In other words, there are other ways to verify network connectivity that don't require you to use MSK – OneCricketeer Aug 11 '23 at 00:36

0 Answers0