A Kafka .NET client, written by Confluent.
Source: https://github.com/confluentinc/confluent-kafka-dotnet
Questions tagged [confluent-kafka-dotnet]
138 questions
0
votes
1 answer
Confluent Kafka missing methods for .NET
I am using Confluent.Kafka package v2.0.2 that I downloaded via NuGet in a .NET project in order to create a Kafka Consumer and I noticed that Confluent's docs (this page for instance) include many methods that this package don't such as: Poll(),…
Guy_g23
- 316
- 2
- 7
0
votes
2 answers
Authorization issue when performing operations on MSK cluster using SASL/SCRAM auth method
I have configured the MSK cluster and allowed public access through SASL/SCRAM authentication method. Now I am facing the issue where I do not have the necessary permissions when using these credentials (specified in the Secrets Manager created with…
hcerim
- 959
- 1
- 11
- 27
0
votes
1 answer
C# Kafka IConsumer's Subscribe method doesn't consume any messages but Assign methods works fine
I'm trying to create a simple Kafka consumer in c# using the below code
private static CancellationTokenSource StartConsumer(IAdminClient client, string topicName)
{
ConsumerConfig config = new()
{
BootstrapServers =…
Harsimranjeet Singh
- 53
- 5
0
votes
2 answers
How to use AvroSerializer without a schema registry
I am trying to write a unit test that verifies that adding a new property to an Avro schema is backwards compatible.
First I took the Avro generated .cs model and saved it as MyModelOld.cs and renamed the class inside to MyModelOld.
Then I re-ran…
JALLRED
- 855
- 1
- 11
- 23
0
votes
2 answers
How to configure a kafka broker in one cluster to publish message
I have two broker and I want to restrict 2nd one with particular messages for particular region.
for example we have multiple region in world to provide our services like china, India
I just wanted to divert specific partner data for India region to…
akshay verma
- 23
- 4
0
votes
2 answers
How to improve Kafka consumer performance
I have a consumer application which I would like to speed up. Most of the time consumer is waiting for http response from the third party system hence I cannot proceed with other messages in queue and lag increases. What would be a recommended way…
user2412672
- 1,459
- 3
- 20
- 36
0
votes
0 answers
C# Kafka: How to Create a NetworkException Error
I am writing an integration test for my app and would like to test my exception-handling functionality. More specifically, I need my Kafka Producer's DeliveryReport to claim that a Kafka NetworkException has occurred -- which will trigger my app's…
Vismark Juarez
- 613
- 4
- 14
- 21
0
votes
0 answers
Confluent-kafka client OutOfOrderSequenceException while Kafka/MSK cluster had security patch applied
When sending data to kafka using an async, idempotent producer, we received errors showing below. We restarted producer applications and resolved an issue. We opened a ticket with Confluent - Ticket Link. They mentioned that it's an existing bug in…
Sam
- 1
0
votes
0 answers
Kafka consumer can't read from multiple partitions at the same time
I use Confluent.Kafka 1.9.2 C# library to create single Kafka consumer for listening topic with several partitions. Currently consumer drain out all messages from first partition and only then goes to next. As I know from KIP, I can avoid such…
Dmitriy Marov
- 39
- 5
0
votes
0 answers
dotNet Confluence Kafka consumer client doesn't read the uncommitted message
There is a weird issue I have fallen into where I have created a kafka consumer in dotNet and have disabled the auto commit
config = new Confluent.Kafka.ConsumerConfig
{
BootstrapServers = "localhost:9092",
…
Bimal Gangawal
- 385
- 2
- 17
0
votes
0 answers
Winstore path and password
I am using Confluent.Kafka library in C#.Net
But while passing schema registry I need to pass keystore path and keystore password
I don't have any idea how to get path and password from Keystore on windows.
amlish
- 3
- 3
0
votes
1 answer
MassTransit Kafka null message handling
In our application there is a possibility that the message on Kafka topic is null, while the key is not empty. We'd like to handle those messages, but they are skipped instead of being consumed. Is there a possibility to consume a message from Kafka…
Isard
- 312
- 1
- 14
0
votes
1 answer
Kafka consumer local batch queue memory leak
My service works with a topic through 10 consumers. Unmanaged memory grows immediately after adding a new batch of messages to a cluster topic (picture 1). Memory continues to grow even after a long period of inactivity.
When I sent 500k messages…
Nordennavic
- 46
- 6
0
votes
1 answer
What is the difference between Kafka connect and Confluent Kafka?
Currently, I have Apache Kafka in Docker. I am using a confluent—Kafka client for connecting the Kafka Topics(Producer and Subscriber) from the the.Net core application. Today I came to know that Kafka connect can be used for this. After all the…
Vanapandi Narayanan
- 79
- 1
- 12
0
votes
1 answer
Create a Kafka Topic from SQL Server Stored Proc?
My .NET application is stored procedure intensive and I need to create a Kafka topic from a stored procedure. I have a key process that is running on stored procedure and at the end of that process, I need to create a Kafka topic to trigger the…
JSLover
- 45
- 7