Stack Overflow
Stack Overflow

Questions tagged [confluent-kafka-dotnet]

A Kafka .NET client, written by Confluent.

Source: https://github.com/confluentinc/confluent-kafka-dotnet

138 questions
2
votes
2 answers

Avro fingerprint schema

I am using Confluent kafka in C# to consume Messages, these messages are formatted as hexadecimal strings from which the Schema fingerprint is extracted. How to get the schema from the schema fingerprint in C#? am I missing something?
Elias Ghali
  • 823
  • 1
  • 13
  • 29
2
votes
1 answer

Connection to external Kafka Server using confluent-kafka-dotnet fails

I need to read Kafka messages with .Net from an external server. As the first step, I have installed Kafka on my local machine and then wrote the .Net code. It worked as wanted. Then, I moved to the cloud but the code did not work. Here is the…
Ziad Salem
  • 496
  • 13
  • 34
2
votes
1 answer

Does Azure Event Hub Limit the Number of Topics?

I am trying to use Azure Event Hubs as a message bus using the Confluent Kafka sdk's. When I started testing my code, it was working, but after I created 10 topics, I can no longer create any new topics. I am using IAdminClient.CreateTopicsAsync…
Eric
  • 2,120
  • 1
  • 17
  • 34
2
votes
0 answers

manually committing offsets in kafka

I have a list of offsets with their corresponding partition and I need to commit them manually. To do so I am looping through the list and assigning partition to the consumer and then seeking to a particular offset. then I am consuming the message…
Pooja Singh
  • 19
  • 5
2
votes
1 answer

How To Implement A Wait Retry Transient Fault Handling Policy?

I am fairly new to Kafka and Polly. I am seeking advice with respect to how to implement failure resiliency when using the Admin Client with Kakfa Confluent .NET client. I am using the Admin Client to create a topic if it does not already exist…
anon_dcs3spp
  • 2,342
  • 2
  • 28
  • 62
2
votes
2 answers

Why does consumer read all messages from Kafka topic after restart?

I use confluent .net client. Subscriber always reads all messages from Kafka topic after restarting(subscriber service restarting). How to commit offset that consumer already achieved and read from it? Maybe some consumer configuration can help...
John
  • 164
  • 9
2
votes
2 answers

Kafka consumer doesn't consume messages from existing topic

I have confluent kafka installed running on docker. In the topic, I have 10 partitions. The problem is that I cannot consume messages from that topic, but I can Produce messages on the topic. I am trying to consume from the topic using C#…
Andrei Ivanov
  • 75
  • 1
  • 7
2
votes
1 answer

What does Deserialization of key mean in Consumer class in Confluent Kafka?

The confluent kafka documentation says, a Consumer class is defined as follows: Class Consumer The consumer class above implements a high-level Apache Kafka consumer (with key and value deserialization). I understand the TKey and…
tubby
  • 2,074
  • 3
  • 33
  • 55
1
vote
0 answers

Azure Eventhub Consume Message throttling

Problem Statement Single consumer is consuming the message from the single topic partition. The consuming topic partition has 8578 messages The observation from Azure Eventhub Metrics, Outgoing message is high and seeing the Throttling Request. This…
Vishnu Kumar K S D
  • 21
  • 4
1
vote
1 answer

Kafka Producer in .Net - SSL Handshake Failed

This is the first time I am trying to connect to Kafka server using Asp.Net console app and I ran into a bunch of issues. I solved most but can't shake this one off. I am testing the producer part and keep getting "SSL Handshake Failed" error. I…
NoBullMan
  • 2,032
  • 5
  • 40
  • 93
1
vote
2 answers

Consume ONLY latest message

I'm programming a Kafka Consumer in an ASP.NET web application, all seems good but every time I run the consumer, it consumes all messages in the topic. Here the Consumer config. ConsumerConfig = new ConsumerConfig { GroupId =…
SerPecchia
  • 83
  • 6
1
vote
0 answers

Strange Kafka Lag Issue

I have a kafka cluster with 3 brokers and 3 zoo-keepers and using strimzi template for openshift 3.11. And I have topics with 20 or 30 partitions. Well I can't diagnose why it is happening exactly why but after some mysterious event some of the…
Eldar
  • 9,781
  • 2
  • 10
  • 35
1
vote
1 answer

Memory leak when I put message from Kafka to list of string

I have a memory leak when I consume from kafka topic and put message into a simple list More details: I created a simple web api empty, where i have hosted service. Inside the hosted service i have a infinity loop which every 15 second repeat some…
Wafeelka
  • 41
  • 1
  • 6
1
vote
0 answers

Bulk Insertion in Kafka .Net Client not waiting for specific time interval

I have a simple Kafka producer. I am trying to send message in bulk after a specific interval of time but it is sending the data to the Kafka as soon as the code call the produceAsync method and not waiting for the specific interval. As per my…
shujaat siddiqui
  • 1,527
  • 1
  • 20
  • 41
1
vote
0 answers

Eventhub Connection Issue from Docker as well in kubernetes

I have a code, which uses the Confluent Kafka library, written in C#. I Am able to connect to the Azure event hub from my local system properly. However, when I put the same in the container. I get the following…
user2010220
  • 11
  • 2
1
2
3
…
9 10