I am trying to add consumers for a topic using kafkajs (v2.0.0) on my node project. i am trying to add a consumer, using example provided in the docs
creating the client:
const kafka = new Kafka({
logLevel: logLevel.DEBUG,
brokers: brokers
})
creating the group
const consumer = kafka.consumer({groupId:'some unique group id'})
followed by async run call to .connect, .subscribe .run
However, i am getting the follwing error when calling .consumer()
{"level":"ERROR","timestamp":"","logger":"kafkajs","message":"[Connection] Response JoinGroup(key: 11, version: 5)","broker":"","clientId":"","error":"The coordinator is not aware of this member","correlationId":,"size":}
in DEBUG:
The group member needs to have a valid member id before actually entering a consumer group
the error is thrown after
[Cluster] Found group coordinator
[Connection] Request JoinGroup
[Connection] Response JoinGroup <----- error
Attempt 1 using default values, passing in a unique groupId, like:
const consumer = kafka.consumer({groupId:'someGroup'})
Actual result:
{"level":"ERROR","timestamp":"","logger":"kafkajs","message":"[Connection] Response JoinGroup(key: 11, version: 5)","broker":"","clientId":"","error":"The coordinator is not aware of this member","correlationId":,"size":}
Attempt 2 Increased session timeout value Decreased heartbeat interval value *kept the heartbeat 1/3 of the session timeout as suggested in the docs
Actual result:
{"level":"ERROR","timestamp":"","logger":"kafkajs","message":"[Connection] Response JoinGroup(key: 11, version: 5)","broker":"","clientId":"","error":"The coordinator is not aware of this member","correlationId":,"size":}
Attempt 3 When i run the consumer via CLI directly on the broker while explicitly setting --group with the group id only then i am able to proggermaticaly call .consumer using the example script w/o seeing this error
Actual result: consumer is added to the previously create group
Attempt 4 i tried the same example using a different library (kafka-node)
Actual result: Consumer group created with no issues.
Any suggestions?
