0

I deployed Strimzi operator for Kafka & enabled TLS/ACLs, now I am stuck! How to run producer.sh/consumer.sh on 9093 passing user created?

Below is the command working with ACLs disabled on port 9092:

kubectl -n myproject run kafka-producer -ti --image=strimzi/kafka:0.14.0-kafka-2.3.0 --rm=true --restart=Never -- bin/kafka-console-producer.sh --broker-list my-cluster-kafka-bootstrap:9092

But if I enable ACL, TLS & use 9093 in the above command then how shall I pass Kafka user details to authenticate?

@Jakub @ppatierno will be glad to receive your help.

ManyThanks, Sudhir

sudhir tataraju
  • 1,159
  • 1
  • 14
  • 30
  • @Jakub could you please help? – sudhir tataraju Apr 16 '20 at 08:35
  • What kind of authentication mechanism did you enabled on the port 9093? Did you created the KafkaUser? – Jakub Apr 16 '20 at 17:19
  • @Jakub Thankyou for responding, enabled TLS & ACL(simple) authorization. I tried this script https://github.com/strimzi/client-examples/blob/master/scripts/run.sh & the generated files copied to /tmp path of kafka strimzi pod & ran producer.sh as in below next comment way but no luck.. getting ANONYMOUS user dont have my-topic permission but the keystore created using hello-world-producer user.crt & user.key who has full permissions to all topics print in acls.sh output. – sudhir tataraju Apr 17 '20 at 08:57
  • /opt/kafka/bin/kafka-console-producer.sh --broker-list my-cluster-kafka-bootstrap:9092 --topic my-topic \ --producer-property security.protocol=SSL \ --producer-property ssl.keystore.type=PKCS12 \ --producer-property ssl.keystore.type=ssl.truststore.type=PKCS12 --producer-property ssl.truststore.password=9949183896 \ --producer-property ssl.truststore.location=/tmp/acheck/truststore.p12 \ --producer-property ssl.keystore.location=/tmp/acheck/keystore.p12 \ --producer-property ssl.keystore.password=9949183896 – sudhir tataraju Apr 17 '20 at 08:57
  • I think the command is corect. You owuld need to connect to port 9093 for TLS of course, not 9092. Does that work for you? If not, what exactly is the error you get? – Jakub Apr 20 '20 at 13:47

3 Answers3

1

you can set it via the CLI script is called kafka-acls.sh

you can find more details from here: https://kafka.apache.org/documentation/#security_authz

-- additional Sorry, I didn't read your question correctly. I am not sure it's working with the Kubernetes command.

S.Lim
  • 62
  • 4
  • you suggested the approach for general kafka configuration but I deployed the kafka using Strimzi operator running on kubernetes, FYI if all configuration changes should be applied over Strimzi cannot be applied directly logging kafka pod, editing config files – sudhir tataraju Apr 16 '20 at 05:38
0

One of the scenario i have seen it as following.

1) I have created 2 new users K1 and K2 with a single topic as T1. 2) Now the relationship between them are:::::K1->T1(Sending) and K2->T1(receiving) 3) So when i launch the console-producer.sh and console-consumer.sh for transporting the message, i have noticed the User:ANONYMOUS is Denied for Describe error.

I was able to resolve this by adding the user:ANONYMOUS to the topic T1 using kafka-acls.sh script.

Some of the Questions i have are::: 1) When i already have K1 and K2 user certs and key files for User authencation, why is it expecting ANONYMOUS as mandatory in place?
2) Do we need add this user for any no of topics we create in future or is there a configuration param in .yaml which can be leveraged to apply automatically?

Viswanadh Peri
  • 1
  • 1
  • 1
  • Is this supposed to be an actual answer or a new question? If it is the latter, please consider opening a new questions. – Michael Heil Apr 29 '20 at 14:05
0

Are you using TLS for encrption or for authentication? I would need to see your Kafka cluster deployment file. If you are using SCRAM-SHA-512 and TLS as encryption, then you will need the add the following --producer.config /tmp/producer.properties, to you your command:

kubectl -n myproject run kafka-producer -ti --image=strimzi/kafka:0.14.0-kafka-2.3.0 --rm=true --restart=Never -- bin/kafka-console-producer.sh --broker-list my-cluster-kafka-bootstrap:9093 --producer.config /tmp/producer.properties

where producer.properties is created inside the pod

  cat /tmp/producer.properties <<EOF
    security.protocol=SASL_SSL
    sasl.mechanism=SCRAM-SHA-512
    sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username=admin password=UGzYA2hddMqz;
    ssl.truststore.location=/tmp/truststore.jks
    ssl.truststore.password=truststorepassword
    EOF

and password is being retrieved by using:

kubectl get secret admin -o jsonpath='{.data.password}' | base64 -d
Oana
  • 537
  • 5
  • 11