kafka consumer AbstractCoordinator: Discovered coordinator Java client

Question

I have 3 brokers running with broker id s 0 1 and 2. The Consumer (Java Client) picks up broker 0 as group coordinator and starts to consume messages correctly. But when the broker 0 which is the group coordinator is down, the consumer does not do anything and stalls on the poll() method. The process resumes only when that broker 0 is up and running.

How to handle this scenario of group co-ordinator change in the Java client?

I get this error when group coordinator dies:

16/09/22 17:42:45 INFO internals.AbstractCoordinator: Discovered coordinator datascience1.sv2.trulia.com:9092 (id: 2147483647 rack: null) for group group2.
16/09/22 17:42:45 INFO internals.AbstractCoordinator: (Re-)joining group group2
16/09/22 17:42:45 INFO internals.AbstractCoordinator: Marking the coordinator datascience1.sv2.trulia.com:9092 (id: 2147483647 rack: null) dead for group group2

I am struggling with the same issue, the producer keeps posting, but the consumer is stalled until coordinator is back. — Pouriya Zarbafian, Sep 19 '17 at 11:08
What kafka version are you using? is the log cleaner enabled? consumer offset topic is a compacted log, if log cleaner is disabled, this could lead to coordinator issues. — Lior Chaga, Feb 20 '18 at 07:08
Does your topic has replication factor 3? Were all replicas synced with the leader? — MeetJoeBlack, Jun 08 '18 at 12:15

Chaouki Dhib · Answer 1 · 2018-09-16T16:09:54.960

0

See this answer: https://stackoverflow.com/a/50954843/7321097 and https://stackoverflow.com/a/50595475/7321097

The isue is in the offsets.topic.replication.factor & replication.factor configs.

edited Sep 16 '18 at 16:09

answered Sep 16 '18 at 15:51

Chaouki Dhib

389
4
17

kafka consumer AbstractCoordinator: Discovered coordinator Java client

1 Answers1