How to disable KAFKA consumer being autostarted without having to make any code changes including setting the autoStartup = "{xyz}" in Spring Boot?

Question

Below is my KAFKA consumer

@Service
public class Consumer {

    private static final Logger LOGGER = Logger.getLogger(Consumer.class.getName());
    public static Queue<ProductKafka> consumeQueue = new LinkedList<>();

    @KafkaListener(topics = "#{'${spring.kafka.topics}'.split('\\\\ ')}", groupId = "#{'${spring.kafka.groupId}'}")
    public void consume(ProductKafka productKafka) throws IOException {
        consumeQueue.add(productKafka);
        LOGGER.info(String.format("#### -> Logger Consumed message -> %s", productKafka.toString()));
        System.out.printf("#### -> Consumed message -> %s", productKafka.toString());
    }
}

and below is my "application.properties" file

spring.kafka.topics=Product
spring.kafka.groupId=Product-Group

My KAFKA consumer is getting started automatically.

However I want to disable KAFKA consumer being autostarted without having to make any changes to the existing code including setting autoStartup = "{xyz}" in the consumer class due to the requirement.

I am looking an existing properties which would disable KAFKA consumer being autostarted, something like this

spring.kafka.consumer.enable=false

Note: I have multiple KAFKA consumers and the above property should disable all the consumers in the project.

do we have any existing properties which would disable KAFKA consumer being autostarted without having to make any changes to the existing code?

Are you trying to programmatically control when consumers start/stop? If so, I'm not sure Spring would be the best option for that level of control — OneCricketeer, Feb 10 '21 at 14:50
No. All I want to do is to disable the KAKFA consumer's auto start behavior using the spring profile property (eg spring.kafka.consumer.enable=false) without having to make any code changes including setting the autoStartup property (eg - @KafkaListener(autoStartup = "{xyz}") — One Developer, Feb 10 '21 at 15:03
I don't want to introduce any custom properties - eg: kafka.consumer.autostart=true and map it to @KafkaListener(autoStartup = "#{'${kafka.consumer.autostart}'}") — One Developer, Feb 10 '21 at 15:12
`> If so, I'm not sure Spring would be the best option for that level of control ` Spring absolutely provides that level of control. https://docs.spring.io/spring-kafka/docs/current/reference/html/#kafkalistener-lifecycle But there is no standard property to control it. — Gary Russell, Feb 10 '21 at 15:25

Gary Russell · Accepted Answer · 2021-02-10T20:26:36.987

2

There is no standard out-of-the-box property; you have to provide your own.

autoStartup="${should.start:true}"

will start the container if property should.start is not present.

EDIT

Just add something like this in your application.

@Component
class Customizer {

    Customizer(AbstractKafkaListenerContainerFactory<?, ?, ?> factory,
            @Value("${start.containers:true}") boolean start) {

        factory.setAutoStartup(start);
    }

}

start:
  containers: false

edited Feb 10 '21 at 20:26

answered Feb 10 '21 at 15:23

Gary Russell

166,535
14
146
179

Is there a way to have a kind of startup class which would read the custom property and control the Kafka autostart? The reason is that there are multiple consumers in the project and there are multiple projects needs to be updated. I don't want to update the individual consumers. – One Developer Feb 10 '21 at 20:03
2

You can set the `autoStartup` property on the container factory and it will apply to all consumers created by it. The annotation property allows it to be overridden for individual listeners. – Gary Russell Feb 10 '21 at 20:05
can you please help me with the sample? – One Developer Feb 10 '21 at 20:15
@GaryRussell - Error - java.lang.IllegalStateException: Failed to load ApplicationContext Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'kafkaConsumerCustomizer' defined in file [kafkaConsumerCustomizer.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.kafka.config.AbstractKafkaListenerContainerFactory' available: expected at least 1 bean which qualifies as autowire candidate. – One Developer Apr 20 '21 at 07:01
@GaryRussell - Can you please suggest? – One Developer Apr 20 '21 at 07:01
It sounds like you are not using Spring Boot, which auto configures that bean. – Gary Russell Apr 20 '21 at 11:50
@GaryRussell - it works fine when I disable the Test cases. I see the above error while test cases are getting executed. I have added the empty default constructor to make the test cases pass however it is always referring the empty default constructor but not the parameterized constructor. Is there a way to use the empty default constructor only for the test cases and parameterized constructor otherwise? – One Developer Apr 20 '21 at 12:20
As I could not edit the question, I have posted the latest code below in the answer section. Both solutions are not helpful, please suggest - @GaryRussell – One Developer Apr 20 '21 at 12:22
Don't add it as an answer; ask a new question instead. – Gary Russell Apr 20 '21 at 12:55
@GaryRussell - As suggested, asked the new question https://stackoverflow.com/questions/67179632/no-qualifying-bean-of-type-org-springframework-kafka-config-abstractkafkalisten. kindly suggest. – One Developer Apr 20 '21 at 13:10

How to disable KAFKA consumer being autostarted without having to make any code changes including setting the autoStartup = "{xyz}" in Spring Boot?

1 Answers1

Linked