0

The Confluent.Kafka AdminClient allows you to create a topic, specifying the name, number of partitions, replication factor, and retention (and I'm guessing other settings through the configs property). The GetMetadata() call, however, returns a TopicMetadata that only has the name and partition information on it. Is there a way to retrieve the replication factor and retention time using the .Net client?

 await adminClient.CreateTopicsAsync(new[]
                    {
                        new TopicSpecification
                        {
                            Name = topicName,
                            NumPartitions = _connectionSettings.TopicAutoCreatePartitionCount,
                            ReplicationFactor = _connectionSettings.TopicAutoCreatePartitionCount,
                            Configs = new Dictionary<string, string> {{"retention.ms", "9999999999999"}}
                        }
                    });
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Derek Greer
  • 15,454
  • 5
  • 45
  • 52
  • 2
    Try this? https://docs.confluent.io/5.5.0/clients/confluent-kafka-dotnet/api/Confluent.Kafka.IAdminClient.html#Confluent_Kafka_IAdminClient_DescribeConfigsAsync_System_Collections_Generic_IEnumerable_ConfigResource__DescribeConfigsOptions_ – OneCricketeer Aug 18 '20 at 17:20
  • Thanks for the comment, OneCricketeer. This set me down the right path. – Derek Greer Aug 22 '20 at 00:55

1 Answers1

1

To get the retention time you can use DescribeConfigsAsync:

var results = await adminClient.DescribeConfigsAsync(new[] { new ConfigResource { Name = "topic_name", Type = ResourceType.Topic } });

foreach (var result in results)
{
    var retentionConfig = result.Entries.SingleOrDefault(e => e.Key == "retention.ms");
}

But I'm not sure what the correct way to get the replication factor is, since it's not retrieved with DescribedConfigsAsync. One way I can think of is to use GetMetadata but it's not a very clean solution:

var meta = adminClient.GetMetadata(TimeSpan.FromSeconds(5));
var topic = meta.Topics.SingleOrDefault(t => t.Topic == "topic_name");
var replicationFactor = topic.Partitions.First().Replicas.Length;
rytisk
  • 1,331
  • 2
  • 12
  • 19
  • 1
    Thanks for providing an answer. I ultimately created a service which utilizes both DescribeConfigsAsync and GetMetadata to retrieve all the properties of a topic I needed for my purposes. I do find it a bit strange that there's a distinction between inherent properties and "metadata". – Derek Greer Aug 22 '20 at 00:55