how to deploy a kafka consumer being in pause mode until i signal to start consume the messages

Question

I am using spring-kafka 2.2.8 and trying to understand if there is an option to deploy a kafka consumer being in pause mode until i signal to start consume the messages. Please suggest.

I see in the below post, we can pause and start the consumer but I need the consumer to be in pause mode when it's deployed. how to pause and resume @KafkaListener using spring-kafka

You could set up an event listener, or some sort of conditional bean to start the consumer when the event happens or a condition is met — Beez, Jun 11 '20 at 16:50

Gary Russell · Accepted Answer · 2020-06-11T17:00:23.210

@KafkaListener(id = "foo", ..., autoStartup = "false")

Then start it using the KafkaListenerEndpointRegistry when you are ready

registry.getListenerContainer("foo").start();

There is not much point in starting it in paused mode, but you can do that...

@SpringBootApplication
public class So62329274Application {

    public static void main(String[] args) {
        SpringApplication.run(So62329274Application.class, args);
    }


    @KafkaListener(id = "so62329274", topics = "so62329274", autoStartup = "false")
    public void listen(String in) {
        System.out.println(in);
    }

    @Bean
    public NewTopic topic() {
        return TopicBuilder.name("so62329274").partitions(1).replicas(1).build();
    }


    @Bean
    public ApplicationRunner runner(KafkaListenerEndpointRegistry registry, KafkaTemplate<String, String> template) {
        return args -> {
            template.send("so62329274", "foo");
            registry.getListenerContainer("so62329274").pause();
            registry.getListenerContainer("so62329274").start();
            System.in.read();
            registry.getListenerContainer("so62329274").resume();
        };
    }

}

You will see a log message like this when the partitions are assigned:

Paused consumer resumed by Kafka due to rebalance; consumer paused again, so the initial poll() will never return any records

awesome, thanks @Gary. This is what I want for some specific requirement. — Raj, Jun 11 '20 at 18:03

how to deploy a kafka consumer being in pause mode until i signal to start consume the messages

1 Answers1