Example to make NestJs Kafka work with Avro?

Question

I am looking for an example to use NestJS Kafka microservice work with Avro schema.

So far the consumers are working with Json Object but we are looking to towards schema registry with Avro as schema type.

Kafka Config:

    transport: Transport.KAFKA,
    options: {
      client: {
        clientId,
        ssl: true,
        brokers,
      },
      consumer: {
        groupId: consumerGroupId,
      },
      subscribe: {
        fromBeginning: false,
      },
    },
  };

Consumer:

@Controller(USER)
@UseInterceptors(ClassSerializerInterceptor)
export class UserController {
  constructor(private readonly service: UserService) {
  }

  @EventPattern(process.env.USER_TOPIC, Transport.KAFKA)
  async processUserEvent(data: UserEvent) {
    await this.service.handleEvent(data);
  }
}

Well, did you read this https://kafka.js.org/docs/introduction#avro ? What exactly is `UserEvent` in your question? — OneCricketeer, Feb 01 '23 at 04:48

OneCricketeer · Answer 1 · 2023-02-01T17:37:55.990

Nest receives incoming Kafka messages as an object with key, value, and headers properties that have values of type Buffer

As does KafkaJS eachMessage handler.

This means you need to get those, rather than "UserEvent", such as using IncomingMessage type mentioned in Nest docs. Then you can follow docs at confluent-schema-registry NPM package

As part of the options, there is a parser setting to "keep binary", but this only applies to the value, which may not work if you also have Avro keys.

Also see https://github.com/nestjs/nest/issues/7288

Example to make NestJs Kafka work with Avro?

1 Answers1