0

I try to send message into right partition here is my NESTJS Producer Controller



 constructor(
    private readonly appService: AppService,
    @Inject('any_name_i_want') private readonly client: ClientKafka,
  ) {}

  async onModuleInit() {
    ['test2'].forEach((key) => this.client.subscribeToResponseOf(`${key}`));
    await this.client.connect();
  }

@Get('kafka-test-with-response')
  async testKafkaWithResponse() {
    const res = await this.client.send('test2', {
      foo: 'bar',
      data: new Date().toString(),
      partition: 1,
    });

    return res;
  }

I try to read in NestJs Document but it's provide link into Kafka website and I've found this in Kafka document

const producer = kafka.producer()

await producer.connect()
await producer.send({
    topic: 'topic-name',
    messages: [
        { key: 'key1', value: 'hello world', partition: 0 },
        { key: 'key2', value: 'hey hey!', partition: 1 }
    ],
})

After I've check in UI Kafka enter image description here

It seems like Producer still send random partition

I try to deep down check in typescript send() function but it use any so I'm really not sure how to specify partition in send function

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
AkwinS
  • 3
  • 2

1 Answers1

0

NestJS does use KafkaJS, so that code section you found is correct, assuming you could get a raw kafkajs client instance.

Alternatively, Kafka uses the record key to partition each event, and in the NestJS ProducerConfig, you can specify a partitioner instance. https://github.com/nestjs/nest/blob/master/packages/microservices/external/kafka.interface.ts#L104

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • thank you very much, Does consumer also can specify read partition too ? – AkwinS Sep 02 '22 at 04:04
  • ConsumerConfig takes a PartitionAssigner - https://github.com/nestjs/nest/blob/master/packages/microservices/external/kafka.interface.ts#L159 But in general, all partitions should be consumed equally. In other words, its a Kafka antipattern to couple the consumers to producers via partitions – OneCricketeer Sep 02 '22 at 14:37