1

I am starting with kafka , have a question on the consumer groups. We have an application where we want different consumers from same group subscribing to different topics. The grouping is done based on some business criteria . To be specific consumer 1 from group A and consumer 2 from group A are subscribed to Topic 1 and Topic 2 each with 10 partitions. Does this mean that there consumer 1 can scale to 10 and consumer 2 also can scale to 10 since they are subscribed to different topics. Is this a correct design

user2254608
  • 33
  • 1
  • 6

1 Answers1

1

yes, since within a topic kafka try to assign partitions to consumers as equal as possible. The key is topic:consumer_group_id so it doesn't matter another_topic:same_consumer_group_id - it's another key, and consumer with topic:consumer_group_id could be scaled to partitions number

J.J. Beam
  • 2,612
  • 2
  • 26
  • 55