KafkaJS is an open-source Node.js client library for Apache Kafka
KafkaJS is an open-source Node.js client library for Apache Kafka, a distributed streaming platform designed to store and process high-throughput data streams.
Questions tagged [kafkajs]
199 questions
0
votes
1 answer
NestJS - Interceptor to log incoming/outgoing kafka events
I have implemented an interceptor in my NestJS application to log http request/responses using axios interceptors
https://github.com/axios/axios#interceptors
Is there something similar that I could use to log incoming/outgoing kafka events in my…
mh377
- 1,656
- 5
- 22
- 41
0
votes
1 answer
kafkajs: Is there a way to create dynamic number of consumer in consumer group?
I am using kafkajs library to implement kafka consumer in nodejs. I tried to find paramater in kafkajs official document where I could specify number of consumers needed in consumer group, but I couldn't find any.
const kafka = await…
Sukhda Jamidar
- 11
- 3
0
votes
0 answers
NodeJs Strimzi: KStream KTable join results in very few key matches
I am using nodefluent/kafka-streams to join a KStream and a KTable. This is with Strimzi on Kubernetes. I have set-up in such a way that, every message in KStream will definitely have a matching key in KTable. However, I see only a few (< 10)…
Chain Head
- 13
- 3
0
votes
0 answers
Poll one message/batch using kafkajs
Kafkajs creates a task that at a determined frequency polls messages from kafka, making it cumbersome to implement backpressure (not impossible).
I wonder if there's an undocumented way of polling messages that allows me to implement something like…
caeus
- 3,084
- 1
- 22
- 36
0
votes
0 answers
Kafkajs Evenemitter consumer.on() not genrating any output
I am trying to create kafka consumer health check using event emmitter. I have tried to implement the following code to check heartbeat and consumer.crash event:
const isHealthy = async function() {
const { HEARTBEAT } = consumer.events;
let…
0
votes
0 answers
How to change client config in kafkajs
As described in a doc https://kafka.js.org/docs/configuration the client must be configured with at least one broker.
const { Kafka } = require('kafkajs')
let kafka = new Kafka({
clientId: 'my-app',
brokers: ['kafka1:9092',…
alphanumeric
- 17,967
- 64
- 244
- 392
0
votes
1 answer
How to send message into custom partition in Kafka with Nest Js
I try to send message into right partition here is my NESTJS Producer Controller
constructor(
private readonly appService: AppService,
@Inject('any_name_i_want') private readonly client: ClientKafka,
) {}
async onModuleInit() {
…
AkwinS
- 3
- 2
0
votes
0 answers
KafkaJS producer and consumer working but not showing the Output
I am trying to execute the below producer code in Node JS using kafkajs module
const {Kafka} = require("kafkajs")
const kafka = new Kafka({
clientId : 'my-producer',
brokers : ['kafka1:9092']
})
const producer = kafka.producer();
const…
Suraj Joshi
- 5
- 2
0
votes
1 answer
Connecting to schema registry using SSL Node.js Kafkajs
I am currently trying to connect to the schema registry in nodejs using kakfa.js as I want to decode messages which consuming from the kafka topic.
Previously I have already used basic authentication (which was with username & password) which was…
0
votes
1 answer
KafkaJS: Commit the whole batch
I'm having trouble committing the whole batch at once processing it. In the docs I see an example where they resolve the offset of a single message from the batch after processing it, I need to resolve the offset or commit them, after processing all…
NorwegianClassic
- 935
- 8
- 26
0
votes
1 answer
Why the service of grpc is undefined when i set configuration for kafka?
i'm trying to implement microservices with grpc and kafka but when i add both options in main.ts the client of grpc doesn't load on "onModuleInit" method
this is my main.ts:
async function bootstrap() {
const app = await…
Andy Guerrero
- 11
- 3
0
votes
1 answer
kafkajs offset management, how to get single message. I looked for solutions everywhere, but nothing seemed to work. Can anybody help me out
I tried all the auto commit options available on kafkajs documentation, here and also manual commits on the same. But nothing seemed to work out for me.
Is there any option that I may be missing excpet those provided in the documentation:…
Chinmaya Patel
- 11
- 1
0
votes
1 answer
Failing to produce to Kafka topic because of 'Failed to connect to seed broker' error with KafkaJS
I have the following file, producer.js, taken from https://kafka.js.org/docs/getting-started.
const { Kafka } = require('kafkajs')
const kafka = new Kafka({
clientId: 'my-app',
brokers: ['broker:9092'],
})
const producer =…
Rahul
- 543
- 1
- 7
- 24
0
votes
0 answers
Kafkajs EVP_DecryptFinal_ex:bad decrypt
I have Kafka producer application which is trying to publish the records to the secured Kafka cluster running on port 9093. The security protocol in use is SSL. I referred this document to do the configurations->…
Swapnil
- 801
- 3
- 19
- 42
0
votes
1 answer
This is not the correct coordinator for this group kafka error
I am using kafkajs and getting this error
{"level":"INFO","timestamp":"2022-07-07T08:55:41.859Z","logger":"kafkajs","message":"[Consumer]…
WISHY
- 11,067
- 25
- 105
- 197