I am trying to split a stream into multiple partition on the basis of a partition key but apparently its not working.
The implementation is such that I have a Class lets say Metrices
public class Metrice {
public string MetriceType { get; set; }
public double MetriceValue { get; set; }
}
The metrices will be consumed asynchronously one at a time and might be having different MetriceType
. What I am trying to achieve is create partition on the basis of MetriceType
. What I have tried so far.
- Setting Message key (partitionKey) as
MetricesType
;
public Message<string, string> FormatMessage(string partitionKey, string message)
{
return new Message<string, string> { Key = partitionKey, Value = message };
}
The messages are always published at Partition.Value
= 0
- Partitoner class in Confluent.Kafka library, was hoping for something similar to Custom Partitoner, this link but couldn't find any .net implementation.
So my question is, is there any way to split my income messages on the basis of a property in this case MetriceType
and publish them on their dedicated partitions (ordering is essential) or the only option I have is to create a Topic using AdminClient
and hardcoding the partition count, or is there an alternative approach I can look into. Thanks in advance.