I've been developing a Kafka stream processing application with the Quarkus-Framework in Java. Now I'm trying to connect to the Kafka brokers via the SASL/PLAIN mechanism, but am getting the following error:
2022-10-27 10:52:06,736 ERROR [org.apa.kaf.cli.NetworkClient] (kafka-admin-client-thread | alarms-preprocessor-dev-a8147d3e-809c-4e96-9ce0-de10e55a8d72-admin) [AdminClient clientId=alarms-preprocessor-dev-a8147d3e-809c-4e96-9ce0-de10e55a8d72-admin] Connection to node -1 (localhost/127.0.0.1:29092) failed authentication due to: Unexpected handshake request with client mechanism PLAIN, enabled mechanisms are []
Apparently, the brokers do not have the PLAIN mechanism enabled, which begs the question why my Kafka-Connect-Service is able to sue the PLAIN-mechanism.
Anyway, this is my broker-configuration (approximately the same for all 3 instances) using confluentinc/cp-kafka docker image with docker-compose:
broker-1:
image: confluentinc/cp-kafka:7.2.1
hostname: broker-1
container_name: broker-1
depends_on:
- zookeeper
ports:
- "29092:29092"
- "9092:9092"
- "9091:9091"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker-1:9092,SASL_PLAINTEXT://broker-1:9091,PLAINTEXT_HOST://localhost:29092
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/kafka/kafka_jaas.conf"
KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
KAFKA_AUTHORIZER_CLASS_NAME: kafka.security.authorizer.AclAuthorizer
KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: "true"
volumes:
- /home/larissa/Projekte/SRE/Kafka/local_dev_cluster/files/kafka_jaas.conf:/etc/kafka/kafka_jaas.conf
and this is part of the output from docker logs broker-1 | grep PLAIN
:
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
advertised.listeners = PLAINTEXT://broker-1:9092,SASL_PLAINTEXT://broker-1:9091,PLAINTEXT_HOST://localhost:29092
inter.broker.listener.name = PLAINTEXT
listener.security.protocol.map = PLAINTEXT:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
listeners = PLAINTEXT://0.0.0.0:9092,SASL_PLAINTEXT://0.0.0.0:9091,PLAINTEXT_HOST://0.0.0.0:29092
sasl.enabled.mechanisms = [PLAIN]
security.inter.broker.protocol = PLAINTEXT
The part that says "sasl.enabled.mechanisms = [PLAIN]" suggests that the PLAIN mechanism is indeed enabled. So maybe it's a problem with my Quarkus application configuration, which looks like this:
quarkus.kafka-streams.application-id=alarms-preprocessor-dev
quarkus.kafka-streams.bootstrap-servers=localhost:29092,localhost:29192,localhost:29292
quarkus.kafka-streams.topics=test_topic
quarkus.kafka-streams.security.protocol=SASL_PLAINTEXT
quarkus.kafka-streams.sasl.mechanism=PLAIN
quarkus.kafka-streams.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="admin" \
password="admin-secret" \
serviceName="alarms-preprocessor";
All JAAS-configs, users and passwords are correct by the way, since they work with Kafka Connect just fine. If necessary, I can provide those too, just let me know.
Thanks in advance for any answers :)