How kafka recognizes consumer group?

Question

I have this code to create consumer balanced in pykafka :

consumer = topic.get_balanced_consumer(consumer_group='Testing',auto_commit_enable=True,zookeeper_connect='amsmgmt002:2181,nas5:2181,amsdblx006:2181')

How the state of consumer is managed across the processes ?

thanks

Hope it helps you : https://kafka.apache.org/documentation/#intro_consumers — Nishu Tayal, Mar 07 '18 at 11:56

score 0 · Answer 1 · answered Apr 09 '18 at 23:14

From the pykafka readme:

You can have as many BalancedConsumer instances consuming a topic as that topic has partitions. If they are all connected to the same zookeeper instance, they will communicate with it to automatically balance the partitions between themselves. The partition assignment strategy used by the BalancedConsumer is the "range" strategy by default. The strategy is switchable via the membership_protocol keyword argument, and can be either an object exposed by pykafka.membershipprotocol or a custom instance of pykafka.membershipprotocol.GroupMembershipProtocol.

How kafka recognizes consumer group?

1 Answers1