0

I am using NodeJS to consume message from Kafka, after received the message , I will bring it to create an index in Elasticsearch. This is my piece of code : 

kafkaConsumer.on('message', function (message) {
    elasticClient.index({
        index: 'test',
        type: 'sample',
        body: message
    }, function (error, response) {
        if (error) {

            // Stop consuming message here

            console.log(error);
        }
        console.log(response);
    });
});

I want to make sure that the index must be created successfully before continue consuming next message, because I don't want any message to be lost.

xtiger
  • 1,446
  • 2
  • 15
  • 33

2 Answers2

0

Have a try consumer.pause()/resume().
pause() Pause the consumer
resume() Resume the consumer

Shawn Guo
  • 3,169
  • 3
  • 21
  • 28
  • can you prove my me a full piece of code to implement this? . I have added consumer.pause() after the if(error) but it does not work. – xtiger May 31 '16 at 16:10
0

You can pause your consumer so that it won't subscribe and fetch new messages. Once you get response from elasticClient you can resume your kafka consumer. 

kafkaConsumer.pause();
kafkaConsumer.resume();

Use it according to your setup.

t6nand
  • 710
  • 1
  • 8
  • 20
  • can you prove my me a full piece of code to implement this? . I have added consumer.pause() after the if(error) but it does not work. – xtiger May 31 '16 at 16:10
  • I think you will have to write your piece of code outside that of error block. Once messages are consumed and you go for creating index and pause your consumer till response is obtained. Thereafter, resume your consumer. You can try something like this: consumer.pause(); if(response) { consumer.resume();} You can try using flags if you try to write functions for them with appropriate callbacks. – t6nand Jun 01 '16 at 06:24