30

I have a problem communicating with Kafka secured with sasl using console scripts. Kafka is secured with sasl, listener is SASL_PLAINTEXT and mechanism is PLAIN.

What I did: I tried listing some data using one of kafka scripts:

bin/kafka-consumer-groups.sh --bootstrap-server (address) --list

However I get 

WARN Bootstrap broker (address) disconnected (org.apache.kafka.clients.NetworkClient)

and command fails, which is understandable because it's secured with sasl.

So I tried how to add client username/password to that command. First, I tried to run kafka-console-consumer script, I used --command-config to add necessary file. I quickly discovered that I can't add jaas file directly and I needed to use .properties file, so I did.

My properties file(keep in mind that brackets indicate "censored" data, I can't put all real data here):

bootstrap.servers=(address)
zookeeper.connect=127.0.0.1:2181
zookeeper.connection.timeout.ms=6000
sasl.jaas.config=(path)/consumer_jaas.conf
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
group.id=(group)

My jaas file:

KafkaClient {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username=(username)
    password=(password);
};

This jaas file works in my standard java applications.

However, when I'm trying to run either kafka-consumer-groups script or kafka-console-consumer, I get this error:

Exception in thread "main" org.apache.kafka.common.KafkaException: java.lang.IllegalArgumentException: Login module not specified in JAAS config
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:94)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:93)
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:51)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:84)
at kafka.admin.AdminClient$.create(AdminClient.scala:229)
at kafka.admin.AdminClient$.create(AdminClient.scala:223)
at kafka.admin.AdminClient$.create(AdminClient.scala:221)
at kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService.createAdminClient(ConsumerGroupCommand.scala:454)
at kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService.<init>(ConsumerGroupCommand.scala:389)
at kafka.admin.ConsumerGroupCommand$.main(ConsumerGroupCommand.scala:65)
at kafka.admin.ConsumerGroupCommand.main(ConsumerGroupCommand.scala)
Caused by: java.lang.IllegalArgumentException: Login module not specified in JAAS config
at org.apache.kafka.common.security.JaasConfig.<init>(JaasConfig.java:68)
at org.apache.kafka.common.security.JaasUtils.jaasConfig(JaasUtils.java:59)
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:85)

This jaas file is a direct copy of a file that I'm using in java app that communicates with kafka and it works, however here, using console tools, it just doesn't work. I tried searching for a solution but I can't find anything useful.

Can anyone help me with this?

Giorgos Myrianthous
  • 36,235
  • 20
  • 134
  • 156
Tomasz
  • 658
  • 1
  • 7
  • 22

1 Answers1

50

There are 2 ways to provide the JAAS configuration to the Kafka clients.

  • Via the client property: sasl.jaas.config. In that case you set it to the actual JAAS configuration entry. For example, your configuration file becomes:

    bootstrap.servers=(address)
zookeeper.connect=127.0.0.1:2181
zookeeper.connection.timeout.ms=6000
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="(username)" password="(password)";
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
group.id=(group)

    As you've already figured out, you can use --command-config to pass a properties file to kafka-consumer-groups.sh.

  • Via the Java property: java.security.auth.login.config. In this case, you set it to the path of your JAAS file. Also if you set it in KAFKA_OPTS, kafka-consumer-groups.sh will pick it up automatically.

    export KAFKA_OPTS="-Djava.security.auth.login.config=(path)/consumer_jaas.conf"
Mickael Maison
  • 25,067
  • 7
  • 71
  • 68
  • I put jaas configuration entry directly into .properties file and it worked. Now I can use these scripts without any errors. I don't know why it had problems with external file. Thank you! – Tomasz Aug 18 '17 at 13:15
  • 8
    When using sasl.jaas.config, you can only set it to the JAAS configuration entry it doesn't work with the path to a file. On the contrary with the Java property, it only works with the path to a file. – Mickael Maison Aug 18 '17 at 13:37
  • 3
    I have set sasl.jaas.config properly, but I still get `[main] ERROR io.confluent.admin.utils.cli.KafkaReadyCommand - Error while running kafka-ready.` and `Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set`. How do I make it stop looking for a .jaases file and instead just use what I've specified? – Patrick Szalapski Sep 23 '22 at 19:25