I have 6 partitions in a topic and I have a nestjs application running on Kubernetes with 6 replicas , Idea is to assign each 1 partition to 1 replica . Whenever I push a new build , I get below error from 3 out of 6 replicas , while other 3 gets the topics assigned .
ERROR [ClientKafka] ERROR [Consumer] Crash: Error: value should be an instance of Encoder {"timestamp":"2022-05-31T09:00:48.617Z","logger":"kafkajs","groupId":"myapp-kafka-staging-client","stack":"Error: value should be an instance of Encoder\n at Encoder.writeEncoder (/home/node_modules/kafkajs/src/protocol/encoder.js:255:13)\n at /home/node_modules/kafkajs/src/protocol/encoder.js:325:18\n at Array.forEach (<anonymous>)\n at Encoder.writeArray (/home/node_modules/kafkajs/src/protocol/encoder.js:316:13)\n at /home/node_modules/kafkajs/src/consumer/assignerProtocol.js:51:44\n at Array.map ()\n at Object.encode (/home/node_modules/kafkajs/src/consumer/assignerProtocol.js:50:33)\n at /home/node_modules/@nestjs/microservices/helpers/kafka-reply-partition-assigner.js:108:78\n at Array.map ()\n at KafkaReplyPartitionAssigner.assign (/home/node_modules/@nestjs/microservices/helpers/kafka-reply-partition-assigner.js:106:40)"}
Here is my client config :
retry: { retries: 8, initialRetryTime: 1, maxRetryTime: 30000, }, connectionTimeout: 10000, authenticationTimeout: 10000,
My consumer config:
maxWaitTimeInMs: 100, sessionTimeout: 120000, heartbeatInterval: 90,
sessionTimeout is quite high as I am consuming some services with higher latency , heartbeatInterval is low as I have kept it below maxWaitTimeInMs and maxWaitTimeInMs is 100 ms as I want to avoid processing concurrent requests .
Please help me out with this issue .
Regards
