Stack Overflow
Stack Overflow

Questions tagged [kafkajs]

KafkaJS is an open-source Node.js client library for Apache Kafka

KafkaJS is an open-source Node.js client library for Apache Kafka, a distributed streaming platform designed to store and process high-throughput data streams.

199 questions
0
votes
0 answers

Thingsboard microservice crashes on AWS EC2 with kafkajs errors

I would like to run Thingsboard in the microservice configuration on a AWS EC2 machine but when I try to launch the services the CPU goes at 100% and the machine crashes. At first I thought that the t3.medium was too small to run all the services…
Manuel Romano
  • 53
  • 6
0
votes
0 answers

How to write unit test cases for kafkajs consumer and producer

I'm new to kafkajs and I want to write unit test cases for the Kafka consumer and producer. Currently my project uses Typescript, Chai and Mocha as test framework.
Kiran
  • 23
  • 4
0
votes
0 answers

is anyone know why we cannot delete a groupid on kafka even if the consumers are not connected

so i use kafka to send messages and receive messages and when it successfully receive a message its going to cleanup tempory create topic and clear the group ids and the topic is going to delete but group id is not can anyone explain or help me my…
ACHIRA GAMING
  • 1
0
votes
0 answers

kafkajs disconnect takes a long time

I use kafkajs for autotests. When calling await consumer.disconnect() - the command takes about 5 seconds, are there any options for a safe and fast disconnect? const consumer = this.client.consumer({ groupId: this.groupId, …
Ruslan
  • 51
  • 4
0
votes
1 answer

How to set reduxjs/toolkit state in eachMessage function of kafkajs using nextjs v13

Messages are coming from kafka and I must set the state of reduxjs/toolkit in kafkajs consumer. I am using nextjs 13 and it has server-side and client-side components. I rehydrated nextjs 13 learned from that video:…
Orhan Gazi
  • 109
  • 1
  • 3
  • 10
0
votes
1 answer

Kafka wiith nestjs error: KafkaJSProtocolError: This server does not host this topic-partition

I have a kafka cluster that I run using docker compose: broker1: image: confluentinc/cp-kafka:7.4.0 hostname: broker1 container_name: broker1 depends_on: - controller ports: - "9092:9092" - "9101:9101" …
Noy Gafni
  • 1,091
  • 9
  • 19
0
votes
0 answers

How to connect docker-compose kafka from localhost?

I have troubles with connection to kafka from localhost. I'm using kafka with KRaft without zookeper. services: kafka_auth: image: confluentinc/cp-kafka:7.2.1 environment: KAFKA_NODE_ID: 2 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP:…
Drew Dru
  • 449
  • 4
  • 16
0
votes
1 answer

KafkaJSProtocolError: Request is not valid given the current SASL state

I have an AWS MKS cluster in which I could create a topic, produce a message and consume a message using installed Kafka in my ec2 server. But when I am trying to use the Kafka producer in my nodejs app, I get the error while npm…
ashique
  • 935
  • 2
  • 8
  • 26
0
votes
0 answers

Kafkajs taking time to consume message in Nodejs

I am using Nodejs package for Kafka consumer. I have 100 topic each having 1 partition. const { Kafka, logLevel } = require("kafkajs"); const kafka = new Kafka({ logLevel: logLevel.INFO, clientId: "kafka9845", brokers:["90.45.78.123"], …
Sudhir
  • 835
  • 11
  • 31
0
votes
0 answers

Can we connect to multiple group Id in kafkajs

We have our kafka consumer application where we have one consumer group and it gets connected to multiple topics using kafkajs node js library const consumeMessages = async() = > { const consumer = kafka.consumer({groupId : 'test-group'}) …
learningmode
  • 137
  • 1
  • 10
0
votes
0 answers

How to implement batch publish in kafka and don't lose data

I'm using kafkajs nodejs library, which doesn't fully implement a batch publish, and from my side I see only one way - is implement it by myself, but I don't fully understand how to do it, so I have some questions How should I calculate message…
mxxnseat
  • 314
  • 1
  • 2
  • 8
0
votes
1 answer

Nodejs Kafka error connection refused on windows 10 machine

I an running kafka broker on my windows 10 machine: [2023-06-19 19:12:33,360] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$) 3.4.1 (Commit:8a516edc2755df89) I am running zookeeper and kafka server…
Anand Vaidya
  • 11
  • 1
0
votes
0 answers

Kafkajs Producer Connection error: connect ECONNREFUSED in localhost

i have this producer const { Kafka } = require("kafkajs"); const kafka = new Kafka({ clientId: "app", brokers: ["localhost:29092"], }); async function sendMessageOnProducer() { const producer = kafka.producer(); await producer.connect(); …
KHALEF Ahmed
  • 36
  • 3
0
votes
1 answer

Kafka broker error: 'There is no leader for this topic-partition as we are in the middle of a leadership election.'

Im running a Kafka Broker Cluster containing 3 brokers. I connected my Nodejs application with cluster. When all three brokers are up, application is connected. everything works fine. but when i shutdown the leader broker in the cluster. then…
Leo Gul
  • 7
  • 1
0
votes
0 answers

How can I rewrite the offset class of Kafka Node in Kafkajs's code?

I want to switch the javascript kafka library from kafka node to kafkajs. I want to find a way to wrap the below code which is the offset module originally from kafka node. How can I rewrite the whole function and also the…
WILLIAM
  • 457
  • 5
  • 28
1 2 3
…
13 14