Apache Kafka vs Oracle Goldengate for remote data streaming

Question

Regarding a data consolidation scenario, are there any comparative analysis that compares Apache Kafka and Oracle Goldengate for remote data streaming?

In the scenario, we have to integrate sensory structured data from multiple (~100) sources to a single destination over internet.

Note: If my question is general, your questions in comment may help me to describe the situation more accurately.

Thanks in advance.

Why have you picked OGG and Apache Kafka as possible candidates; is one already in use? — Robin Moffatt, Jan 08 '18 at 09:36
@RobinMoffatt Our raw data source is sensors, but the streaming tool read them from a RDBMS — Soheil, Jan 08 '18 at 09:46
@RobinMoffatt They are only our candidates for implementation, better option will be considered. The choice is open yet — Soheil, Jan 08 '18 at 10:09
@RobinMoffatt A central database (RDBMS). For another options such as HDFS the choice is different? — Soheil, Jan 10 '18 at 11:21
What is this question about? Comparing Apache Kafka vs GoldenGate? Please descibe the configuration that we are comparing — Adam Leszczyński, May 25 '20 at 18:32

score 3 · Accepted Answer · answered Feb 18 '18 at 11:12

3

Presuming the 100 sources are all Oracle Databases then Oracle Goldengate would be the best choice if you want to configure vs code. You can use Oracle Goldengate to "replicate" the 100 sources to a single location for processing.

Also remember that Oracle Goldengate and Kafka are not either or, Oracle Goldengate can deliver the changes to a Kafka topic.

answered Feb 18 '18 at 11:12

Joe

46
2

What does this mean? "are not either or, Oracle Goldengate can deliver the changes to a Kafka topic." – KON May 09 '21 at 20:31
@KON it means that they do not exclude each other, they can be used in conjunction – German Mar 08 '22 at 21:41

dcjgg · Answer 2 · 2018-05-05T21:20:09.847

You can use Oracle GG for Big Data (Oracle GG) in conjunction with the Kafka client to deliver messages to Kafka topics or whatever the final destination is. One advantage of using Oracle GG for Big Data is you can use the above link to download a full version for prototyping and POC.

This would hold true for whatever GoldenGate flavor of RDBMS or java based solution you are investigating. Oracle GG can gather the data from many heterogeneous sources and pump the data to the consolidation point.

You can deliver the data over the internet via encrypted trail to the target and hand-off to the Kafka client. It really depends on the topology you choose to deliver the data from source to target.

Another consideration is Oracle might be able to provide a cloud based replication service for your business to deliver the data to its destination.

score 0 · Answer 3 · answered Mar 29 '18 at 07:23

Striim actually has the same change data capture capabilities as GoldenGolden to capture data from oracle...AND uses Kafka as the underlying message bus. You can read a tutorial on how to do CDC from a database and stream the data to kafka here:

http://www.striim.com/blog/2018/03/tutorial-real-time-database-integration-apache-kafka-change-data-capture/

disclaimer: i do work at striim

Apache Kafka vs Oracle Goldengate for remote data streaming

3 Answers3