0

I'm currently facing the following issue when using AWS MSK Connector (Debezium Postgres Connector)

[Worker-0509fac07b9701a23] [2022-01-19 04:55:28,759] ERROR Failed testing connection for jdbc:postgresql://debezium-cdc.fac07b9701a2.ap-south-1.rds.amazonaws.com:5432/ecommerce with user 'debezium' (io.debezium.connector.postgresql.PostgresConnector:133)

I've test AWS MSK Connector using Kafka Clients on EC2, I'm able to produce & consume messages. I've also setup AWS MSK S3 Sink Connector, that is working as well.

I've double checked the security groups config for AWS RDS, I'm able to connect to it from EC2.

I'm not sure whats causing this issue.

Here's the Connector Configuration

connector.class=io.debezium.connector.postgresql.PostgresConnector
tasks.max=1
database.hostname=debezium-cdc.fac07b9701a2.ap-south-1.rds.amazonaws.com
database.port=5432
database.dbname=ecommerce
database.user=debezium
database.password=password
database.history.kafka.bootstrap.servers=b-2.awskafkatutorialclust.awskaf.c4.kafka.ap-south-1.amazonaws.com:9094,b1.awskafkatutorialclust.awskaf.c4.kafka.ap-south-1.amazonaws.com:9094,b-3.awskafkatutorialclust.awskaf.c4.kafka.ap-south-1.amazonaws.com:9094
database.server.id=1
database.server.name=debezium-cdc
database.whitelist=ecommerce
database.history.kafka.topic=dbhistory.ecommerce
include.schema.changes=true
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
James Z
  • 12,209
  • 10
  • 24
  • 44
hbhargav
  • 1
  • 2

1 Answers1

3

You need to set up AWS RDS Database Publicly accessible: No. Because your AWS MSK is in a private network (VPC) and it can not connect to public Databases (Read more: https://docs.aws.amazon.com/vpc/latest/userguide/how-it-works.html).

Please try to change your RDS Database Postgres Publicly accessible: No. And create MSK connect again. (make sure that your AWS RDS Database is the same VPC, Security Group as your AWS MSK.)

Anyway, If you want to connect with your private AWS RDS Database, you need to do about bastion host (Read more: https://aws.amazon.com/premiumsupport/knowledge-center/rds-connect-ec2-bastion-host/).

enter image description here

Thawatchai
  • 133
  • 1
  • 7