Stack Overflow
Stack Overflow

Questions tagged [change-data-capture]

Change data capture (CDC) encompasses database design patterns to keep track of changed data and perform actions with it.

In databases, change data capture (CDC) is a set of software design patterns used to determine (and track) the data that has changed so that action can be taken using the changed data. Also, Change data capture (CDC) is an approach to data integration that is based on the identification, capture and delivery of the changes made to enterprise data sources.

CDC solutions occur most often in data-warehouse environments since capturing and preserving the state of data across time is one of the core functions of a data warehouse, but CDC can be utilized in any database or data repository system.

271 questions
1
vote
2 answers

T-SQL Capture Minimum Log Sequence Number of A Table At A Specific Time Using Change Data Capture

I have two tables, call them dbo.T1 and dbo.T2. I want to capture delta every few hours using CDC. The first time I query CDC, I pass in sys.fn_cdc_get_min_lsn('dbo_t1') ( and sys.fn_cdc_get_min_lsn('dbo_t2') ) as @from_lsn for…
007
  • 2,136
  • 4
  • 26
  • 46
1
vote
1 answer

How to guarantee the topic events order generated by debezium, stored in kafka and sent to spark?

I am working in a change data capture project. I have a mysql database. I use debezium to capture all the changes and send it to kafka. And later I read all the info from Spark and I send it to Apache Phoenix using jdbc. I am using debezium with a…
David Zamora
  • 383
  • 1
  • 4
  • 15
1
vote
2 answers

Change data capture in snowflake

I was wondering if anyone knows if snowflake can do change data capture, and how it would be done. I can't seem to find any documentation on it. Thanks
2tone_tony
  • 47
  • 1
  • 10
1
vote
0 answers

How to embed debezium in a java application to capture Oracle change data?

I am using debezium in my java application to capture changes from Oracle 12c. The oracle database is accessible on localhost:1521. Below is respective java code. // Define the configuration for the embedded and Oracle connector ... …
chathuranga siriwardhana
  • 33
  • 5
1
vote
2 answers

Why does Change Data Capture (SQL Server) job auto stop after specific duration?

I'm currently synchronizing data between Maria <--> MSSQL. That is 2 way sync. I used SQL Server on Windows and everything works well until several days before... I switched all the test DB to Linux server, so MSSQL was run on a Docker container…
Neo.Mxn0
  • 953
  • 2
  • 8
  • 25
1
vote
0 answers

Sync data from sql server to postgresql database in Java

I coded a java server synchronize data from oracle database to postgresql database used a technology name Oracle DCN. Now I want do a sync like it but to use sql server to postgresql. Noted sql server provided change tracking and Change Data Capture…
invlong
  • 11
  • 3
1
vote
0 answers

How implement correctly MERGE for tables? Incorrect CDC result

Once of a day one local SQL server doing synchronization with some linked server. The following query using for merge: MERGE [TargetDb].[dbo].[TargetTable] AS Target USING (SELECT * FROM [RemoteServer].[SourceDb].[dbo].[TestTable]) AS Source ON…
user6408649
  • 1,227
  • 3
  • 16
  • 40
1
vote
0 answers

How to get data changes from SQL to another system?

My current system, I'll call it B, gets data from an old legacy SQL database, A, via replication. I do not have any control over this legacy system other than access to the database. Replication was initially chosen to keep the two systems separate…
bearDog
  • 31
  • 3
1
vote
1 answer

How to capture data in mysql with debezium change data capture and consume with jdbc sink in kafka connect?

I have problem of capturing data in mysql with debezium change data capture and consuming it to another mysql using kafka connect jdbc sink. Because the schema and payload that debezium produces to kafka topic is not compatible with the schema that…
Jason Foster
  • 117
  • 10
1
vote
1 answer

SQL Server 2008 Database Secondary

We have an older vendor supplied application that is earmarked for platform upgrades in 2019 but is currently running SQL Server 2008 (SP4). It's about 1.2TB of data. Our internal IT unit has come to the point that we want to create a readable…
Dan White
  • 11
  • 3
1
vote
2 answers

SQL Server 2008 CDC Non-responsive & Stuck After Table Dropped

I was testing out the SQL Server 2008 change data capture feature on a table when it became stuck. There is only one table enabled for CDC. After performing various DDL changes to the table to test how CDC responds, I dropped the table. Previously…
ulty4life
  • 2,972
  • 1
  • 25
  • 31
1
vote
2 answers

on what Sql Server product levels the Change Data Capture (CDC) feature is available?

I have a project, which involves using Sql Server 2012 Standard Edition. This project needs to have Sql Server Change Data Capture enabled (CDC). Is CDC available in Sql Server Standard Edition?
Eugene Goldberg
  • 14,286
  • 20
  • 94
  • 167
1
vote
0 answers

Change Data Capture (CDC)

I was reading this article about ETL for analytics databases, and I came across this interesting note: If you discover that your internal applications are deleting data that’s important for analysis, you have two options: either ask your …
Michael Lee
  • 458
  • 1
  • 8
  • 18
1
vote
1 answer

Does increasing of a field size require disabling and enabling cdc for a table?

I have to increase the size of a table field from 255 to 4000 chars. CDC is enabled for that table. Before I alter the column, should I disable cdc for the table or is the change picked-up automatically? I read this link:…
boggy
  • 3,674
  • 3
  • 33
  • 56
1
vote
0 answers

Overcome CDC drawback

My company would like to make use of Change Data Capture to replace Interchange of Interface Files between Upstream System and Downstream Systems. Upstream System runs in Oracle Database and contains superset of data while Downstream Systems run in…
Stanley Stein
  • 397
  • 1
  • 3
  • 17
1 2 3
…
18 19