Stack Overflow
Stack Overflow

Questions tagged [confluent-platform]

a Streaming Platform providing enterprise event streaming functionalities complementing Apache Kafka.

Confluent Platform provides enterprise scaled event streaming expanding . It adds specific server infrastructure, network operability, storage solutions, administrative tools, and deployment staging, among others.

Official website: www.confluent.io

2076 questions
52
votes
1 answer

confluent platform vs apache kafka

I am new to kafka and I am curious about the Confluent platform. It seems that there are not many user stories on the Confluent platform. What are the differences between the Confluent platform and Apache Kafka?
Jihun No
  • 1,201
  • 1
  • 14
  • 29
47
votes
8 answers

Confluent Maven repository not working?

I need to use the Confluent kafka-avro-serializer Maven artifact. From the official guide I should add this repository to my Maven pom confluent http://packages.confluent.io/maven/ The problem is…
gvdm
  • 3,006
  • 5
  • 35
  • 73
31
votes
2 answers

Ideal value for Kafka Connect Distributed tasks.max configuration setting?

I am looking to productionize and deploy my Kafka Connect application. However, there are two questions I have about the tasks.max setting which is required and of high importance but details are vague for what to actually set this value to. If I…
PhillipAMann
  • 887
  • 1
  • 10
  • 19
26
votes
3 answers

Kafka setup with docker-compose

Hi I'm currently setting up Kafka with Docker. I've managed to setup Zookeeper and Kafka with the published confluent image, see following docker-compose file: version: '2' services: zookeeper: image: confluentinc/cp-zookeeper:3.2.0 …
Omar Andres Olivares Rodriguez
  • 97
  • 2
  • 6
  • 12
20
votes
1 answer

Kafka producer difference between flush and poll

We have a Kafka consumer which will read messages and do so stuff and again publish to Kafka topic using below script producer config : { "bootstrap.servers": "localhost:9092" } I haven't configured any other configuration like…
shakeel
  • 801
  • 1
  • 8
  • 24
20
votes
1 answer

How to pass parameters for a specific Schema registry when using Kafka Avro Console Consumer?

I am trying to use Confluent kafka-avro-console-consumer, but how to pass parameters for Schema Registry to it?
Joe
  • 11,983
  • 31
  • 109
  • 183
20
votes
4 answers

How to have KafkaProducer to use a mock Schema Registry for testing?

I'm using KafkaProducer in my test cases and my producer uses the schemaRegistryUrl which points a my local instance of Schema Registry. Is there a way to mock how KafkaProducer connects with the Schema Registry? That is, to have…
Glide
  • 20,235
  • 26
  • 86
  • 135
17
votes
1 answer

Kafka-connect, Bootstrap broker disconnected

Im trying to setup Kafka Connect with the intent of running a ElasticsearchSinkConnector. The Kafka-setup, consisting of 3 brokers secured using Kerberos, SSL and and ACL. So far Ive been experimenting with running the connect-framework and the…
Jiinxy
  • 559
  • 2
  • 5
  • 17
15
votes
1 answer

Kafka - min.insync.replicas interpretation

I am going through the documentation looking at multiple places, it is adding up confusion.. About the property min.insync.replicas When a producer sets acks to "all" (or "-1"), this configuration specifies the minimum number of replicas that…
Nag
  • 1,818
  • 3
  • 24
  • 41
15
votes
1 answer

Kafka consumer gets stuck after exceeding max.poll.interval.ms

When the consumer does not receives a message for 5 mins (default value of max.poll.interval.ms 300000ms) the consumer comes to a halt without exiting the program. The consumer process hangs and does not consume any more messages. The following…
esquarer
  • 373
  • 1
  • 4
  • 12
15
votes
2 answers

KafkaAvroDeserializer does not return SpecificRecord but returns GenericRecord

My KafkaProducer is able to use KafkaAvroSerializer to serialize objects to my topic. However, KafkaConsumer.poll() returns deserialized GenericRecord instead of my serialized class. MyKafkaProducer KafkaProducer producer; …
Glide
  • 20,235
  • 26
  • 86
  • 135
14
votes
1 answer

How to Consume from specific TopicPartitionOffset with Confluent.Kafka in .Net

I need my consumer to consume from an specific TopicPartitionOffset(here from offset 278). Suppose that Messages have been produced by some Producer in Specific topic like ="Test_1" before. Here is my Code using System; using…
Hassan Ahmadi
  • 626
  • 5
  • 14
13
votes
2 answers

Use Confluent Hub without Confluent Platform installation

I am using these installation instructions for getting confluent hub client https://docs.confluent.io/current/connect/managing/confluent-hub/client.html But, when I get to the line to install the kafka-connect-elasticsearch connector confluent-hub…
dennismonsewicz
  • 25,132
  • 33
  • 116
  • 189
13
votes
3 answers

How to create a Kafka Topic using Confluent.Kafka .Net Client

It seems like most popular .net client for Kafka (https://github.com/confluentinc/confluent-kafka-dotnet) is missing methods to setup and create Topics. When calling Producer.ProduceAsync() the topic is created automatically but I can't find a way…
Michael D.
  • 1,249
  • 2
  • 25
  • 44
13
votes
6 answers

Start Confluent Schema Registry in windows

I have windows environment and my own set of kafka and zookeeper running. To use custom objects, I started to use Avro. But I needed to get the registry started. Downloaded Confluent platform and ran this: $ ./bin/schema-registry-start…
user1860447
  • 1,316
  • 8
  • 25
  • 46
1
2 3
…
99 100