6

I'm using Kafka 2.1 in SASL_PLAINTEXT mode and for controlling ACLs on topics I need to create users and groups. So how can I create a user in running cluster?

I know that I can use kafka-consumer-groups.sh to create and list groups. but when I try to list groups it throws Failed to find brokers to send ListGroups exception.

Amin
  • 975
  • 8
  • 24
  • Any suggestion would be welcome! – Amin Dec 10 '18 at 08:16
  • 1
    2.11 sounds like a Scala version. The exact Kafka version might be more helpful (the latest one is 2.1.0). Also, what kind of SASL mechanism are you using for authentication? PLAIN? SCRAM-SHA? – Jakub Dec 10 '18 at 08:46
  • @Jakub I updated the post. you're right, the Kafka version is 2.1.0 and SASL mechanism is PLAIN. – Amin Dec 10 '18 at 09:48

2 Answers2

8

The users (and their passwords) for the SASL PLAIN mechanism are configured in a JAAS configuration file. Assuming you have multiple Kafka nodes, you should keep this file in sync on all of them. Some example of how to configure it can be found here in the Kafka Docs. So unless you have some shared storage for this file, you might need to modify it multiple times.

You might want to consider using SCRAM instead of PLAIN. SCRAM stores the credentials (usernames and the hashes of their passwords) in Zookeeper and you can just change it using one of the Kafka utilities (more details again in the Kafka docs)

Just to be clear, the consumer groups are not related to user groups for ACL purposes. They are used to group the consumers to distribute the message load among them (by defining which consumers is assigned which partitions) and to store the last consumed offsets. I do not think the SASL PLAIN mechanism has any support for traditional user groups. I think that in most cases you don't really create these groups - they are created when the consumer starts using them.

The only way the consumer groups relate to ACLs is by allowing the different users to use the consumer groups. If you enable the SimpleAclAuthorizer, you can use the kafka-acls.sh utility to manage the ACLs. one of the permissions you can give to the users is the permissions to consume messages using a consumer group. More details about the ACLs and example how to use the kafka-acls.sh tool are in the Kafka Docs.

Jakub
  • 3,506
  • 12
  • 20
0

a pure java solution that worked for me:

public static void main(String[] args) throws NoSuchAlgorithmException {
    String zookeeperHost = "localhost:2181";
    int sessionTimeoutMs = 300000;
    int connectionTimeoutMs = 15000;
    boolean isSecure = false;
    int maxInFlightRequests = 10;
    Time time = Time.SYSTEM;
    String metricGroup = "myGroup";
    String metricType = "myType";
    String username = "alice";
    String password = "alice-secret";


    KafkaZkClient zkClient = KafkaZkClient.apply(zookeeperHost, isSecure, sessionTimeoutMs, connectionTimeoutMs,
      maxInFlightRequests, time, metricGroup, metricType, null, Option.apply(new ZKClientConfig()));

    AdminZkClient adminZkClient = new AdminZkClient(zkClient);

    HashMap<String, String> userProp = new HashMap<>();
    ScramCredential scramCredential = new ScramFormatter(ScramMechanism.SCRAM_SHA_256).generateCredential(password, 4096);
    userProp.put(userSecureSchema, ScramCredentialUtils.credentialToString(scramCredential));

    Properties configs = adminZkClient.fetchEntityConfig(ConfigType.User(), username);
    configs.putAll(userProp);
    adminZkClient.changeConfigs(ConfigType.User(), username, configs);
}

you need SASL_SCRAM enabled on kafka. and you need org.apache.kafka:kafka_2.13:2.5.0 in dependency

Lao Dao
  • 1