1

I am trying to configure MSK connect in AWS and the below is the configuration.

INFO [AdminClient clientId=adminclient-1] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager:235)
[Worker-02003b81ffe0ee9c3] [2022-06-02 14:26:40,955] INFO [AdminClient clientId=adminclient-1] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager:235)

[Worker-02003b81ffe0ee9c3] org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1654180000954, tries=1, nextAllowedTryMs=1654180001055) timed out at 1654180000955 after 1 attempt(s)

As per https://aws.amazon.com/premiumsupport/knowledge-center/msk-connector-connect-errors/ I have opened all traffic for the MSK connector to be able to reach the msk cluster, yet I notice timeout errors.

The connector and the cluster are both in same subnets and uses same security group ID. I ma able to telnet to the broker from a VM in the same subnet.

Note: I have plaintext enabled and no authentication. I have also given proper IAM permission and role attached. This is verified.

Vishnu
  • 11
  • 1
  • 3

1 Answers1

0

Adding the solution in case if it helps someone. https://docs.aws.amazon.com/msk/latest/developerguide/mkc-tutorial-setup.html

I had to create a vpc endpoint as mentioned in the above doc and also associate the subnet route tables that my kafka uses.

Additionally also make sure your SG's have correct inbound and outbound rules

Vishnu
  • 11
  • 1
  • 3