I'm trying to do change data capture with Debezium using Postgres, Kafka, Kafka connect and debezium Postgres connector. Having an issue when trying to start Kafka Connect service with Debezium-Postgres-connector.
This is the plugin.path in my config/connect-standalone.properties:
plugin.path=/opt/kafka/kafka_2.13-3.1.0/connect/debezium-connector-postgres/
The connect-debezium-postgres.properties file:
name=first-connector
connector.class=io.debezium.connector.postgresql.PostgresConnector
database.hostname=postgres
database.port=5432
database.user=postgres
database.password=password
database.server.id=1
database.server.name=bankserver1
database.include.list=bank
table.inlcude.list=bank.holding
database.history.kafka.bootstrap.servers=localhost:9092
database.history.kafka.topic=dbhistory.test
include.schema.changes=true
tombstones.on.delete=false
The command when starting Kafka Connect service with Debezium-Postgres-Connector:
bin/connect-standalone.sh config/connect-standalone.properties config/connect-debezium-postgres.properties
The error Kafka Connect service:
[2022-02-22 10:41:50,571] ERROR Failed to create job for config/connect-debezium-postgres.properties (org.apache.kafka.connect.cli.ConnectStandalone:107)
[2022-02-22 10:41:50,581] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:117)
java.util.concurrent.ExecutionException: org.apache.kafka.connect.runtime.rest.errors.BadRequestException: Connector configuration is invalid and contains the following 1 error(s):
A value is required
You can also find the above list of errors at the endpoint `/connector-plugins/{connectorType}/config/validate`
at org.apache.kafka.connect.util.ConvertingFutureCallback.result(ConvertingFutureCallback.java:115)
at org.apache.kafka.connect.util.ConvertingFutureCallback.get(ConvertingFutureCallback.java:99)
at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:114)
Caused by: org.apache.kafka.connect.runtime.rest.errors.BadRequestException: Connector configuration is invalid and contains the following 1 error(s):
A value is required
You can also find the above list of errors at the endpoint `/connector-plugins/{connectorType}/config/validate`
at org.apache.kafka.connect.runtime.AbstractHerder.maybeAddConfigErrors(AbstractHerder.java:691)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:207)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.lambda$null$0(StandaloneHerder.java:193)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
[2022-02-22 10:41:50,597] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect:67)
[2022-02-22 10:41:50,597] INFO Stopping REST server (org.apache.kafka.connect.runtime.rest.RestServer:311)
[2022-02-22 10:41:50,630] INFO Stopped http_8083@26be6ca7{HTTP/1.1, (http/1.1)}{0.0.0.0:8083} (org.eclipse.jetty.server.AbstractConnector:381)
[2022-02-22 10:41:50,631] INFO node0 Stopped scavenging (org.eclipse.jetty.server.session:149)
[2022-02-22 10:41:50,646] INFO REST server stopped (org.apache.kafka.connect.runtime.rest.RestServer:328)
[2022-02-22 10:41:50,646] INFO Herder stopping (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:106)
[2022-02-22 10:41:50,649] INFO Worker stopping (org.apache.kafka.connect.runtime.Worker:199)
[2022-02-22 10:41:50,650] INFO Stopped FileOffsetBackingStore (org.apache.kafka.connect.storage.FileOffsetBackingStore:66)
[2022-02-22 10:41:50,650] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics:659)
[2022-02-22 10:41:50,651] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics:663)
[2022-02-22 10:41:50,651] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics:669)
[2022-02-22 10:41:50,652] INFO App info kafka.connect for 10.0.2.15:8083 unregistered (org.apache.kafka.common.utils.AppInfoParser:83)
[2022-02-22 10:41:50,652] INFO Worker stopped (org.apache.kafka.connect.runtime.Worker:220)
[2022-02-22 10:41:50,663] INFO Herder stopped (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:124)
[2022-02-22 10:41:50,664] INFO Kafka Connect stopped (org.apache.kafka.connect.runtime.Connect:7
