A Kafka .NET client, written by Confluent.
Source: https://github.com/confluentinc/confluent-kafka-dotnet
Questions tagged [confluent-kafka-dotnet]
138 questions
0
votes
0 answers
Kafka C#: Error occured: Subscribed topic not available: unknown topic or partition
I am trying to run producer/consumer/ktable code in kafka and I keep getting this error message even though the topics are the same. It was working 2 days ago and I haven't touched my code since but then it suddenly stopped working and gave me this…
anon
- 1
- 1
0
votes
0 answers
Kafka consumer messages best practices
Net client. I have below implementation for consumer.
while (true)
{
try
{
_logger.LogInformation("started consuming");
var consumeResult =…
Niranjan
- 63
- 5
0
votes
1 answer
Confluent Kafka: How to replicate partitioner hashing function so that it's possible for the consumer to consume the correct partition?
I have set up a Kafka server with 1000 partitions and 1 Topic. Basically, the data that gets stored in the cluster are status updates of objects. There are about 17000 of these objects and I produce maybe 5 status updates per second. Every object…
atthijs98
- 15
- 1
- 4
0
votes
0 answers
Error producing message Local Value serialization error in Kafka
Hi I am working in Kafka and .Net. I am trying to produce message as below. I have created configuration for schema registry and producer as below. I am using certificates to connect to schema registry
var schemaRegistryConfig = new…
Niranjan
- 63
- 5
0
votes
0 answers
Kafka Consumer does not receive data
I have worked out a Kafka concept which is built up in such a way that I have a series of machines which have electric airpressure dustextraction and machinenStatus data. I have set up my Kafka concept in such a way that I create a topic for each…
tebrex_rtk
- 1
- 2
0
votes
0 answers
Unable to produce messages using kafka producer with dot net
I am trying to implement kafka producer and consumer with dot net running through docker but it is not producing messages to topic and giving this error....my broker is running and even I am able to implement the same in cmd but not through dot net…
Aditi Singri
- 1
- 1
0
votes
1 answer
Simple Confluent Kafka .net client fails to connect to simple build of AK Kafka broker
Any hints on how to get a simple .Net Kafka client to connect to a broker?
I built Apache Kafka from the 3.4.0 download and followed the quick start successfully (topic created, events produced, events consumed).
Using Confluent.Kafka v2.1.1 I…
me_online
- 371
- 3
- 11
0
votes
1 answer
How do I make the C# Kafka client consume the latest topic message?
I created a C# Kafa client with Confluent.Kafka, and I have been trying to read the latest message only from two different topics, but it reads all messages, not just the latest one.
Here is my kafka client class, I'm not sure what I'm missing…
Ahmed Elshorbagy
- 350
- 1
- 5
- 16
0
votes
0 answers
How to come out of while loop once all messages consumed in kafka?
net core console application. I have below implementation of kafka consumer
var config = new ConsumerConfig
{
GroupId = groupId,
BootstrapServers = brokerList,
SaslMechanism = SaslMechanism.Plain,
SaslUsername = saslUsername,
…
Niranjan
- 63
- 5
0
votes
0 answers
Kafka producer "MessageMaxBytes" property needs to be larger than max.message.bytes value on the broker?
We encountered an issue where a message we sent to the topic resulted in the following error from the Producer code:
Confluent.Kafka.ProduceException`2[System.String,com.capgroup.comet.compliance.ComplianceResponse]: Broker: Message size too…
firstblud
- 135
- 3
- 11
0
votes
0 answers
Consume a messa Kafka net core 6
im new with Kafka and im studing now, and i get some issue when im try consume a message. I have 2 example, when i create a user and when create a new Product. For User the message is consumed correclty in my debug, but for product dosen't work.
the…
cesarpowerr20
- 41
- 6
0
votes
0 answers
Confluent Kafka Consumer.Consume() not able to catch any exceptions ( invalid credentials)
using System;
using System.Collections.Generic;
using System.Net;
using System.Security.Authentication;
using System.Threading.Tasks;
using Confluent.Kafka;
class program
{
static async Task Main(string[] args)
{
try
{
…
0
votes
2 answers
How can I consume just the last (newest) message from a Kafka topic with Kafka Confluent Consumer in C#?
I have following code for reading data from a Kafka topic. My target is periodically to read just the last newest message in the topic because I want use the data in a live chart. I have written the following code. But if I run the code I start…
Mdarende
- 469
- 3
- 13
0
votes
1 answer
Kafka producer in C# as Windows Service
I created a Kafka producer, using Confluent library, as a console app that ran as a scheduled task to submit some error records to Kafka broker periodically. Worked fine.
I have to convert this to run as a Windows Service and I am running into some…
NoBullMan
- 2,032
- 5
- 40
- 93
0
votes
0 answers
How to Kafka Custom Practitioner
I have come across one scenario where I want to maintain single kafka topic(message-processing-topic) with 'N' Partitions. Say for example 200 partitions.
I want to use the same topic for all the customers. Say I have 20 customers. I want to use 10…
Vishnu Kumar K S D
- 21
- 4