Stack Overflow
Stack Overflow

Questions tagged [debezium-engine]

30 questions
0
votes
2 answers

Can there be 2 Debezium connector running in one Kafka cluster?

From Debezium official page, there's this picture showing that multiple Debezium connector can connect to the same Kafka. So I have 2 databases, 2 Debeziums, 1 Kafka running in docker-compose, but it seems like only 1 debezium sent update to kafka…
Jackk-Doe
  • 109
  • 7
0
votes
0 answers

debezium mysql 2.1.2 data latency is stable at 1 minute

When I use debezium mysql 2.1.2, by observing the message timestamp in kafka and comparing it with the writing time of the data in the database, I found that the change will be sent to kafka after 1 minute after the data in the database is written.…
pedoc
  • 1
  • 1
0
votes
0 answers

debezium 2.0 connector sqlserver issue in mBean, JmxAttributeGauge

In older versions of Debezium using SQL Server as an example, metrics were available using the following naming strategy: debezium.sql_server:type=connector-metrics,server=,context= In new release debezium 2.0, the…
Govind Yadav
  • 33
  • 4
0
votes
0 answers

Debezium server fffset file storage claim or cleanup

So, for non-kafka deployment, we specify offset storage file with debezium.source.offset.storage.file.filename as per documentation, how does the storage is reclaimed or cleanup happens? I am planning to have k8s deployment with mounted volume…
Yogen Rai
  • 2,961
  • 3
  • 25
  • 37
0
votes
1 answer

Debezium - Oracle Connector - Service Not Starting

DebeziumEngine looking for kafka topic eventhough i have not specified KafkaOffsetBackingStore for offset.storage Reference : DebeziumEngine Config Config Configuration config = Configuration.create() .with("name",…
Praveen D
  • 1
0
votes
1 answer

debezium connector failover mechanism

i'm learning about debezium connectors and im using debezium for postgresql. I have a small question to clarify. Imagine a situation like this. I have a debezium connector for a table called tableA and changes happening on that table publish to a…
Dushan
  • 1,365
  • 20
  • 26
0
votes
0 answers

DebeziumEngine throws SQLNonTransientConnectionException on close

Spring boot app, Embedded DebeziumEngine, MySql connector. when closing application (ctrl+c), Im closing Debezium Engine (DebeziumEngine.close) within @PreDestroy, after few seconds, connectors throws…
ofer.ignite
  • 1
  • 1
0
votes
0 answers

Debezium Server source connector for GCP cloud SQL Server without Kafka

I am trying to use Debezium server (Standalone without Kafka) to load data (incremental) from GCP Cloud SQL Server to PubSub and then BigQuery. I am running a test on my laptop to see if that's doable. I set up debezium server on docker this is my…
Ariox66
  • 620
  • 2
  • 9
  • 29
0
votes
0 answers

Debezium sql connector status api is not showing exception

we are trying to upgrade debezium-connector-sqlserver-1.0.0. to debezium-connector-sqlserver-1.9.5..After upgrade ,the debezium connector status api always shows running status ,even if connector has exception.the server has adopt open jdk…
Shala
  • 31
  • 4
0
votes
0 answers

Task becomes UNASSIGNED for Debezium MySQL source connector

I am using debezium 1.9. I created a connector using below config { "name": "user_management_db-connector-5", "config": { "connector.class": "io.debezium.connector.mysql.MySqlConnector", "tasks.max": "1", …
Partha
  • 814
  • 6
  • 13
0
votes
1 answer

How Debezium connectors handles errors?

I am planning to use MySQL Debezium connector and have a question on the configuration for error handling. Apache Kafka Connect have seems to have errors.tolerance property for error handling. And it is well explained in blog I am trying to…
JDev
  • 1,662
  • 4
  • 25
  • 55
0
votes
2 answers

Debezium Postgres Connector "After applying the include/exclude list filters, no changes will be captured"

I am using Debezium (a Kafka Connector) to capture Postgres database changes and I am getting an error from Debezium. Does anyone know what the error below means and perhaps offer a suggestion to fix it. A bit more debugging info: I tried both…
Eric Broda
  • 6,701
  • 6
  • 48
  • 72
0
votes
1 answer

Using Debezium with AWS Documentdb

Hi all I am trying to use debezium mongodb connector with aws documentdb. Since document db don't have oplog whether debezium will work with Aws document db change streams?
Justin
  • 17
  • 4
0
votes
0 answers

Why postgres drops the logical replication slot own its own after running sometime?

Here I am trying to running the embedded Debezium engine using springboot which connect to postgress database. When the service run it able to connect to the database and also create the replication slot, but after running for sometime, the…
abhi bajaj
  • 1
0
votes
2 answers

Debezium / kafka connect neither creating topics nor pushing data to created topic

I have zookeeper, kafka, and kafka connect running with a debezium plugin. I cannot see any errors. I have whitelisted a db and a couple of tables. Upon inserts or updates to the tables no topic is created; and no data dir for the topics in…
JL_SO
  • 1,742
  • 1
  • 25
  • 38
1
2