How to throw custom error for KafkaJS client when initialization fails?

Question

I am using the following code to initialize the Kafka client:

this.kafka = new Kafka({
  clientId: <my_client_ID>,
  brokers: [
    `${process.env.KAFKA_BROKER_1}`,
    `${process.env.KAFKA_BROKER_2}`,
    `${process.env.KAFKA_BROKER_3}`,
  ],
  retry: {
    initialRetryTime: 3000,
    retries: 3,
  },

});

Now if there's an issue with connecting to the brokers it will throw errors like this:

{"level":"ERROR","timestamp":"2022-10-19T04:21:08.143Z","logger":"kafkajs","message":"[Connection] Connection timeout","broker":"<broker_1>","clientId":"<my_client_id"}
{"level":"ERROR","timestamp":"2022-10-19T04:21:08.144Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection timeout","retryCount":0,"retryTime":299}
{"level":"ERROR","timestamp":"2022-10-19T04:21:08.143Z","logger":"kafkajs","message":"[Connection] Connection timeout","broker":"<broker_2>","clientId":"<my_client_id"}
{"level":"ERROR","timestamp":"2022-10-19T04:21:09.447Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection timeout","retryCount":1,"retryTime":564}
{"level":"ERROR","timestamp":"2022-10-19T04:21:08.143Z","logger":"kafkajs","message":"[Connection] Connection timeout","broker":"<broker_3>","clientId":"<my_client_id"}
{"level":"ERROR","timestamp":"2022-10-19T04:21:11.014Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection timeout","retryCount":2,"retryTime":1008}

Now, I want to change the log message here OR add a custom message after this happens. I am thinking of wrapping it in a try-catch block but for some reason it's not throwing the exception. So:

Is there a way to change this out of the box log to a custom log?
How can I add a custom long if (1) is not possible if the Kafka client fails to initialize?

Thanks.

score 0 · Answer 1 · answered Nov 09 '22 at 08:43

From what I understand, you want to add custom logs to handle certain error scenarios.

Kafkajs provides a way to create our own custom logs and use them instead of default ones.

here is the reference for the same. You can check your version of kafkajs to avoid compatibility issues. Have added an example below.

{
        level: 4,
        label: 'INFO', // NOTHING, ERROR, WARN, INFO, or DEBUG
        timestamp: '2017-12-29T13:39:54.575Z',
        logger: 'kafkajs',
        message: 'Started',
        // ... any other extra key provided to the log function
    }

const { logLevel } = require('kafkajs')
const winston = require('winston')
const toWinstonLogLevel = level => switch(level) {
    case logLevel.ERROR:
    case logLevel.NOTHING:
        return 'error'
    case logLevel.WARN:
        return 'warn'
    case logLevel.INFO:
        return 'info'
    case logLevel.DEBUG:
        return 'debug'
}

const WinstonLogCreator = logLevel => {
    const logger = winston.createLogger({
        level: toWinstonLogLevel(logLevel),
        transports: [
            new winston.transports.Console(),
            new winston.transports.File({ filename: 'myapp.log' })
        ]
    })

    return ({ namespace, level, label, log }) => {
        const { message, ...extra } = log
        logger.log({
            level: toWinstonLogLevel(level),
            message,
            extra,
        })
    }
}

const kafka = new Kafka({
    clientId: 'my-app',
    brokers: ['kafka1:9092', 'kafka2:9092'],
    logLevel: logLevel.ERROR,
    logCreator: WinstonLogCreator
})

Thanks.

How to throw custom error for KafkaJS client when initialization fails?

1 Answers1