KafkaJS is an open-source Node.js client library for Apache Kafka
KafkaJS is an open-source Node.js client library for Apache Kafka, a distributed streaming platform designed to store and process high-throughput data streams.
Questions tagged [kafkajs]
199 questions
0
votes
1 answer
Time limit for pausing kafka consumer
I wanted to implement a functionality which requires Kafka queue to be paused and resume, what I want's to know that is there any time limit upto which it can be paused?
mohit
- 91
- 6
0
votes
1 answer
How can I periodically get new access_token using sasl with oauthbearer for kafka producer and consumer using kafkajs before the token expires?
const axios = require('axios');
const dotenv = require('dotenv');
dotenv.config();
const { Kafka } = require('kafkajs');
var qs = require('qs');
var data = qs.stringify({
'client_id': process.env.CLIENT_ID,
'client_secret':…
Atiq Baqi
- 612
- 1
- 7
- 16
0
votes
0 answers
UnhandledPromiseRejectionWarning when returning from a custom ExceptionFilter with Kafka and NestJS
I'm building a NestJS microservice that receives incoming messages via Kafka (from another NestJS application).
I've created an ExceptionFilter to handle all errors and return them via Kafka.
@Catch(KafkaException)
export class…
PedroSG
- 466
- 2
- 9
- 38
0
votes
0 answers
Use Kafka with NodeJs
I run and setuping lots of projects related to Kafka but I am facing lots of issues and errors. I created a producer-consumer repo. but it didn't work properly. I will make one application.in that web-app I will post from one account to another…
0
votes
1 answer
Why are Kafka messages out of order when we only have a single partition?
We're using KafkaJS to send messages to a Kafka topic that is being consumed by a single consumer. Our Kafka instance is configured in server.properties to only have a single partition, and we can confirm the topic only has a single partition by…
Caleb Koch
- 656
- 1
- 9
- 15
0
votes
0 answers
KafkaJS - Currently I am working on execute Payment in background using Kafka, I want to configuare consumer to execute single message at a time
KafkaJS - Currently I am working on execute Payment in background using Kafka, I want to configuare consumer to execute single message at a time.
I tried maxBytes But Couldn't succedd with this config
Ketan
- 54
- 3
0
votes
1 answer
How to get all messages from a kafka?thread - NestJS (Node)
I have a kafka application running on docker, and I have my nestjs app as microserver, I would like to have a route that takes all the existing messages from a topic inside the kafka broker, is there a way?
Below is my nestjs code, if you can help…
Christian Guimarães
- 105
- 10
0
votes
0 answers
Maximum call stack size exceeded while running jest (kafkajs)
After configuring Kafkajs in node, integration tests are returning "Maximum call stack size exceeded" error while running jest.
KafkaClient:
export const createKafkaClient = () => {
const kafka = new Kafka({
clientId,
brokers,
logLevel:…
0
votes
0 answers
Multiple ssl certificates in Kafkajs
I have a requirement wherein I need to import multiple ssl certificates for my kafka consumer/producer.
I tried the following
ssl: {
rejectUnauthorized: false,
ca: [fs.readFileSync(path.join(__dirname, '..',…
Anish
- 745
- 2
- 8
- 21
0
votes
0 answers
Checking Kafka Consmer for receving the emitted event in Nestjs
I have a bunch of Nestjs microservices running using Kafka, ex. Auth server and API-Gateway.
API-Gateway-> app.module.ts:
@Module({
imports: [
ConfigModule.forRoot({
isGlobal: true,
validationSchema: Joi.object({
…
B Sai Uttej
- 11
- 2
0
votes
0 answers
How does fetch works in KafkaJS?
I am looking at NestJs Kafka integration/KafkaJs and have a few questions around how it's working.
The way I have defined the consumer is as below:
@EventPattern(process.env.TOPIC_NAME, Transport.KAFKA)
async processEvent(event: Record
JDev
- 1,662
- 4
- 25
- 55
0
votes
0 answers
Socket.IO-Client in angular not receiving all the messages sent by server(Kafka-consumer) node.js
I am building an Apache Kafka application where Producer (built on Dotnet Api) successfully published 90,045 messages to topic and Consumer (Built on Node.js) has received all of them. But when consumer is sending those messages to Angular UI…
Samal
- 1
- 1
0
votes
0 answers
kafka not receiving message from topics
I have 3 topics with 1 partition each and 1 consumer group with 3 members and 1 broker also.
Below are the topics
topic1 (with partition=1)
topic2 (with partition=1)
topic3 (with partition=1)
Below are the members with subscribed topic
member1…
AMit SiNgh
- 325
- 4
- 17
0
votes
0 answers
kafka freezes if request hangs
we have 3 microservices - a, b and c.
a needs response from b,
b in order to give response to a, needs response from c.
whenever c isn't available, b freezes, not only on current endpoint, but even on others, not related to microservice c
we use…
Tvuan
- 1
- 1
0
votes
1 answer
NestJS Kafka e2e test examples
Does anybody know how to write e2e tests for Kafka to test producing/consuming events ?.
I have had a look at the NestJS github repo below and cant find anything.
https://github.com/nestjs/nest/tree/master/sample
mh377
- 1,656
- 5
- 22
- 41