I have a scenario where we are forwarding our application logs to Kafka topic using fluentD agents, as Kafka team introduced Kerberos authentication and fluentD version not supporting this authentication, I cannot directly use forward logs. Now we have introduced a new Kafka server without authentication and created a topic there, I want forward messages from this topic in the new server to another topic in another server using Kafka connectors, want to know how I can achieve this?
Asked
Active
Viewed 2,870 times
1 Answers
4
There's several different tools that enable you to stream messages from a Kafka topic on one cluster to a different cluster, including:
- MirrorMaker (open source, part of Apache Kafka)
- Confluent's Replicator (commercial tool, 30 day free trial)
- uReplicator (open sourced from Uber)
- Mirus (open sourced from Salesforce)
- Brucke (open source)
Disclaimer: I work for Confluent.
Robin Moffatt
- 30,382
- 3
- 65
- 92
-
From what I can find looking over sources/issues of the open source projects, it seems only Replicator will copy over the topic configs along with just the data like all the others. Correct me if I'm wrong. – OneCricketeer Dec 13 '18 at 23:44