Spring Cloud Kafka and AWS Secrets 'inputBindingLifecycle' exception

Question

I am trying to integrate our spring cloud kafka apps with aws secrets on spring boot 2.4 and Spring cloud 2020.0.1

Here is my test code:

@SpringBootApplication
public class DemoApplication {

    @Value(value = "${secret-property}")
    private String awsSecretProperty;

    // Test AWS Secrets
    @PostConstruct
    public void testProp() {
        System.out.println("AWS Secret Property: " + awsSecretProperty);
    }

    // Test Kafka
    @Bean
    public Consumer<String> sink() {
        return message -> {
            System.out.println("Message: " + message);
        };
    }

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

}

My properties are:

spring.application.name=test-kafka-aws-secrets

spring.config.import=aws-secretsmanager:test-secret
aws.secretsmanager.enabled=true

spring.cloud.function.definition=sink
spring.cloud.stream.default-binder=kafka

spring.kafka.consumer.key-deserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer
spring.kafka.consumer.value-deserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer

spring.cloud.stream.bindings.sink-in-0.destination=test-topic
spring.cloud.stream.bindings.sink-in-0.group=${spring.application.name}
spring.cloud.stream.kafka.bindings.sink-in-0.consumer.startOffset=earliest
spring.cloud.stream.bindings.sink-in-0.content-type=application/*+avro
spring.cloud.stream.bindings.sink-in-0.consumer.use-native-decoding=true

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.properties.schema.registry.url=http://localhost:8081

I have these two dependencies in my pom:

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-stream-kafka</artifactId>
        </dependency>

        <dependency>
            <groupId>io.awspring.cloud</groupId>
            <artifactId>spring-cloud-starter-aws-secrets-manager-config</artifactId>
            <version>2.3.0-RC2</version>
        </dependency>

I get the following exception when I run the above code:

org.springframework.context.ApplicationContextException: Failed to start bean 'inputBindingLifecycle'; nested exception is java.lang.IllegalStateException: Could not register object [com.amazonaws.services.secretsmanager.AWSSecretsManagerClient@388be5fd] under bean name 'configDataAWSSecretsManager': there is already object [com.amazonaws.services.secretsmanager.AWSSecretsManagerClient@388be5fd] bound
    at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:181) ~[spring-context-5.3.4.jar:5.3.4]
    at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54) ~[spring-context-5.3.4.jar:5.3.4]
    at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356) ~[spring-context-5.3.4.jar:5.3.4]
    at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[na:na]
    at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155) ~[spring-context-5.3.4.jar:5.3.4]
    at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123) ~[spring-context-5.3.4.jar:5.3.4]
    at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:934) ~[spring-context-5.3.4.jar:5.3.4]
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:585) ~[spring-context-5.3.4.jar:5.3.4]
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:767) ~[spring-boot-2.4.3.jar:2.4.3]
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:759) ~[spring-boot-2.4.3.jar:2.4.3]
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:426) ~[spring-boot-2.4.3.jar:2.4.3]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:326) ~[spring-boot-2.4.3.jar:2.4.3]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1311) ~[spring-boot-2.4.3.jar:2.4.3]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1300) ~[spring-boot-2.4.3.jar:2.4.3]
    at com.example.demo.DemoApplication.main(DemoApplication.java:31) ~[classes/:na]
Caused by: java.lang.IllegalStateException: Could not register object [com.amazonaws.services.secretsmanager.AWSSecretsManagerClient@388be5fd] under bean name 'configDataAWSSecretsManager': there is already object [com.amazonaws.services.secretsmanager.AWSSecretsManagerClient@388be5fd] bound
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.registerSingleton(DefaultSingletonBeanRegistry.java:124) ~[spring-beans-5.3.4.jar:5.3.4]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.registerSingleton(DefaultListableBeanFactory.java:1138) ~[spring-beans-5.3.4.jar:5.3.4]
    at io.awspring.cloud.autoconfigure.secretsmanager.AwsSecretsManagerConfigDataLocationResolver.lambda$registerAndPromoteBean$1(AwsSecretsManagerConfigDataLocationResolver.java:113) ~[spring-cloud-starter-aws-secrets-manager-config-2.3.0-RC2.jar:2.3.0-RC2]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176) ~[spring-context-5.3.4.jar:5.3.4]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169) ~[spring-context-5.3.4.jar:5.3.4]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143) ~[spring-context-5.3.4.jar:5.3.4]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:131) ~[spring-context-5.3.4.jar:5.3.4]
    at org.springframework.boot.DefaultBootstrapContext.close(DefaultBootstrapContext.java:133) ~[spring-boot-2.4.3.jar:2.4.3]
    at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:392) ~[spring-boot-2.4.3.jar:2.4.3]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:325) ~[spring-boot-2.4.3.jar:2.4.3]
    at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:144) ~[spring-boot-2.4.3.jar:2.4.3]
    at org.springframework.cloud.stream.binder.DefaultBinderFactory.getBinderInstance(DefaultBinderFactory.java:326) ~[spring-cloud-stream-3.1.1.jar:3.1.1]
    at org.springframework.cloud.stream.binder.DefaultBinderFactory.doGetBinder(DefaultBinderFactory.java:215) ~[spring-cloud-stream-3.1.1.jar:3.1.1]
    at org.springframework.cloud.stream.binder.DefaultBinderFactory.getBinder(DefaultBinderFactory.java:143) ~[spring-cloud-stream-3.1.1.jar:3.1.1]
    at org.springframework.cloud.stream.binding.BindingService.getBinder(BindingService.java:379) ~[spring-cloud-stream-3.1.1.jar:3.1.1]
    at org.springframework.cloud.stream.binding.BindingService.bindConsumer(BindingService.java:102) ~[spring-cloud-stream-3.1.1.jar:3.1.1]
    at org.springframework.cloud.stream.binding.AbstractBindableProxyFactory.createAndBindInputs(AbstractBindableProxyFactory.java:112) ~[spring-cloud-stream-3.1.1.jar:3.1.1]
    at org.springframework.cloud.stream.binding.InputBindingLifecycle.doStartWithBindable(InputBindingLifecycle.java:58) ~[spring-cloud-stream-3.1.1.jar:3.1.1]
    at java.base/java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608) ~[na:na]
    at org.springframework.cloud.stream.binding.AbstractBindingLifecycle.start(AbstractBindingLifecycle.java:57) ~[spring-cloud-stream-3.1.1.jar:3.1.1]
    at org.springframework.cloud.stream.binding.InputBindingLifecycle.start(InputBindingLifecycle.java:34) ~[spring-cloud-stream-3.1.1.jar:3.1.1]
    at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178) ~[spring-context-5.3.4.jar:5.3.4]
    ... 14 common frames omitted

My whole code is essentially above. Any idea how I could resolve this error?

Thank you!

Neil Brown · Answer 1 · 2021-02-26T10:53:27.750

It would be useful to see the full exception stack trace. Based on the first line of the stack trace you've provided to date, an IllegalStateException with the same cause as the one you've mentioned is thrown when the Spring Boot (2.4) app configuration defines the spring.config.import property (with the same prefix, in this case "aws-secretsmanager:") more than once, in different property sources. For example for a Spring Boot 2.4 on application startup -

:: Spring Boot ::                (v2.4.2)

2021-02-25 21:18:33.155  INFO 88284 --- [           main] o.s.b.c.c.ConfigDataLocationResolver     : Loading secrets from AWS Secret Manager secret with name: /my-secret, optional: false
2021-02-25 21:18:33.192 ERROR 88284 --- [           main] o.s.boot.SpringApplication               : Application run failed

java.lang.IllegalStateException: Could not register object [com.amazonaws.services.secretsmanager.AWSSecretsManagerClient@4d21c56e] under bean name 'configDataAWSSecretsManager': there is already object [com.amazonaws.services.secretsmanager.AWSSecretsManagerClient@4d21c56e] bound
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.registerSingleton(DefaultSingletonBeanRegistry.java:124) ~[spring-beans-5.3.3.jar:5.3.3]
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.registerSingleton(DefaultListableBeanFactory.java:1138) ~[spring-beans-5.3.3.jar:5.3.3]
        at io.awspring.cloud.autoconfigure.secretsmanager.AwsSecretsManagerConfigDataLocationResolver.lambda$registerAndPromoteBean$1(AwsSecretsManagerConfigDataLocationResolver.java:113) ~[spring-cloud-starter-aws-secrets-manager-config-2.3.0-RC2.jar:2.3.0-RC2]
        at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176) ~[spring-context-5.3.3.jar:5.3.3]
        at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169) ~[spring-context-5.3.3.jar:5.3.3]
        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143) ~[spring-context-5.3.3.jar:5.3.3]
        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:131) ~[spring-context-5.3.3.jar:5.3.3]
        at org.springframework.boot.DefaultBootstrapContext.close(DefaultBootstrapContext.java:133) ~[spring-boot-2.4.2.jar:2.4.2]
        at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:392) ~[spring-boot-2.4.2.jar:2.4.2]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:325) ~[spring-boot-2.4.2.jar:2.4.2]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1311) ~[spring-boot-2.4.2.jar:2.4.2]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1300) ~[spring-boot-2.4.2.jar:2.4.2]
        at com.brighttalk.service.securedsecrets.Application.main(Application.java:43) ~[main/:na]

The example code you've provided above suggests this is not the case (its a single properties file with one definition of this property). But you might want to double-check your code is not loading properties from another source also. Regards.

If I run he code without the Spring Cloud Kafka Stream, the AWS Secrets Manager works and vice versa. Combined, they are causing the above error. — Katanic, Feb 26 '21 at 14:29

Spring Cloud Kafka and AWS Secrets 'inputBindingLifecycle' exception

1 Answers1