0

[Question posted by a user on YugabyteDB Community Slack]

I have the YB Debezium running and I see that it has access to the table. here is the log:

2022-07-27 21:25:56,894 INFO AbstractConfig values:
 (org.apache.kafka.common.config.AbstractConfig) [pool-3-thread-1]
2022-07-27 21:27:55,147 INFO Retrieving configuration from Secret debezium-secret in namespace default (io.strimzi.kafka.AbstractKubernetesConfigProvider) [pool-3-thread-2]
2022-07-27 21:27:55,197 INFO Successfully tested connection for jdbc:postgresql://someip:5433/testdb with user 'yugabyte' (io.debezium.connector.yugabytedb.YugabyteDBConnector) [pool-3-thread-2]
2022-07-27 21:27:55,198 INFO Connection gracefully closed (io.debezium.jdbc.JdbcConnection) [pool-25-thread-1]
2022-07-27 21:27:55,202 INFO Discovered tablet YB Master for table YB Master with partition ["", "") (org.yb.client.AsyncYBClient) [New I/O worker #16]
2022-07-27 21:27:55,204 INFO Adding table 00004000000030008000000000004001 for streaming (testdb.public.test) (io.debezium.connector.yugabytedb.YugabyteDBConnector) [pool-3-thread-2]
2022-07-27 21:27:55,205 WARN Filtering out the table **** since it was not in the include list (io.debezium.connector.yugabytedb.YugabyteDBConnector) [pool-3-thread-2]
2022-07-27 21:27:55,207 INFO AbstractConfig values:
 (org.apache.kafka.common.config.AbstractConfig) [pool-3-thread-2]

But streamzi operator log shows this. I cannot figure out what’s going wrong!

io.strimzi.operator.cluster.operator.assembly.ConnectRestException: PUT /connectors/debezium-connector-yugabyte/config returned 400 (Bad Request): Connector configuration is invalid and contains the following 1 error(s):
A value is required
You can also find the above list of errors at the endpoint `/connector-plugins/{connectorType}/config/validate`
        at io.strimzi.operator.cluster.operator.assembly.KafkaConnectApiImpl.lambda$createOrUpdatePutRequest$1(KafkaConnectApiImpl.java:96) ~[io.strimzi.cluster-operator-0.29.0.jar:0.29.0]
        at io.vertx.core.impl.future.FutureImpl$1.onSuccess(FutureImpl.java:91) ~[io.vertx.vertx-core-4.2.4.jar:4.2.4]
        at io.vertx.core.impl.future.FutureBase.lambda$emitSuccess$0(FutureBase.java:54) ~[io.vertx.vertx-core-4.2.4.jar:4.2.4]

Does this mean something needed to be implemented on YB connector that is not?

dh YB
  • 965
  • 3
  • 10

1 Answers1

0

At least from the snippet of YB Debezium logs, it looks like it has initialised correctly. However the streamzi logs do not mention which key is missing. (The log line is from within Kafka Connect code). The snapshot feature is not out yet and we are working on it so perhaps that's why you might be facing the error with snapshot. For now, only snapshot.mode=never is implemented and we will soon be releasing the support for other snapshot modes.

Edit:

The connector with snapshot mode initial has been released now. You can find it on GitHub under releases: https://github.com/yugabyte/debezium-connector-yugabytedb/releases/tag/v1.7.0.11-BETA With this, once you start streaming with snapshot.mode:initial the connector will first take a snapshot of the existing data in the table and once the snapshot is complete, it will stop (You’ll get to know this by the logs). After it stops, you will need to restart the connector with snapshot.mode:never to stream further changes.

dh YB
  • 965
  • 3
  • 10