How to integrate Camel Spring boot with Kafka in Confluent Cloud?

Question

I have an application which uses camel Spring Boot with debeezium to listen to a mysql database and to publish in a Kafka topic.

It was all working fine since I changed the kafka from local to use confluent cloud. I have some other applications (normal producers and consumers) that connect to confluent cloud and all works fine.

This is my application.yml. I removed the debeezium-mysql part because it works fine. So I left only the config part of Kafka/Confluent.

routes:
  debezium:
    allow-public-key-retrieval: true
    bootstrap-servers: ${application.kafka.brokers}
    offset-storage:
      topic-cleanup-policy: compact

camel:
  component:
    debezium-mysql:
     ALL CONFIG with mysql, it is not here because it is working fine
    kafka:
      brokers: ${application.kafka.brokers}
      schema-registry-u-r-l: ${application.schema-registry.base-urls}
      value-serializer: io.confluent.kafka.serializers.KafkaAvroSerializer
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      additional-properties:
         #CCloud Schema Registry Connection parameter
        schema.registry.basic.auth.credentials.source: USER_INFO
        schema.registry.basic.auth.user.info: ${SCHEMA_REGISTRY_ACCESS_KEY}:${SCHEMA_REGISTRY_SECRET_KEY}
        ssl.endpoint.identification.algorithm: https
        client.dns.lookup: use_all_dns_ips
      sasl-jaas-config: org.apache.kafka.common.security.plain.PlainLoginModule required username="${CONFLUENT_CLOUD_USERNAME}" password="${CONFLUENT_CLOUD_PASSWORD}";
      security-protocol: SASL_SSL
      retry-backoff-ms: 500
      request-timeout-ms: 20000
      sasl-mechanism: PLAIN

With this config, it keeps giving me an error when I try to star the app:

[AdminClient clientId=adminclient-1] Node -1 disconnected.
2023-02-02 16:57:24.853  INFO 9644 --- [| adminclient-1] org.apache.kafka.clients.NetworkClient   : [AdminClient clientId=adminclient-1] Cancelled in-flight API_VERSIONS request with correlation id 0 due to node -1 being disconnected (elapsed time since creation: 146ms, elapsed time since send: 146ms, request timeout: 3600000ms)

I could check that the problem is in this AdminConfig, which doesn't get the right properties. An example, the security.protocol should be SASL_SSL, but it gets PLAINTEXT. But when creating the producer and consumer it gets the right values.

Really, I have been struggling with this two days. I would be really happy with any help. Thank you.

How to integrate Camel Spring boot with Kafka in Confluent Cloud?

0 Answers0