I'm trying to create a simple Kafka consumer in c# using the below code
private static CancellationTokenSource StartConsumer(IAdminClient client, string topicName)
{
ConsumerConfig config = new()
{
BootstrapServers = BootstrapServers,
GroupId = "testConsumerGroup",
AutoOffsetReset = AutoOffsetReset.Earliest,
};
IConsumer<Null, string> consumer = new ConsumerBuilder<Null, string>(config).Build();
//consumer.Assign(new TopicPartitionOffset(topicName, 0, Offset.Beginning));//this works
consumer.Subscribe(topicName);//this doesn't work
CancellationTokenSource cancellationTokenSource = new();
CancellationToken cancellationToken = cancellationTokenSource.Token;
Task.Run(() =>
{
while (!cancellationToken.IsCancellationRequested)
{
ConsumeResult<Null, string> consumeResult = consumer.Consume();
Console.WriteLine($"{consumeResult.Offset}: {consumeResult.Message.Value}");
}
consumer.Close();
consumer.Dispose();
});
return cancellationTokenSource;
}
When I use the Assign method, my consumer is able to consume the messages just fine. But when I try to use the Subscribe method, my consumer is not able to consume any messages. The method consumer.Consume() never returns.
When I tried to debugging I found that after calling the consumer.Subscribe(topicName) the consumer.Assignment list is empty. Based on this https://github.com/confluentinc/confluent-kafka-dotnet/issues/278 my guess is that for some reason the coordinator is not assigning any partition to my consumer.
I'm creating the topic like this
private static async Task<string> CreateTopic(IAdminClient client, string topicName)
{
await client.CreateTopicsAsync(new TopicSpecification[]
{
new TopicSpecification()
{
Name = topicName,
ReplicationFactor = 1,
NumPartitions = 1
}
});
return topicName;
}
System Details
- OS = Windows 10
- Kafka Version = 3.4.0
- Java = jdk 1.8.0_202 (32 bit)
- Confluent.Kafka NuGet = 2.0.2
