0

I have enabled Kerberos from Ambari v2.7.3 and HDP v3.1.0 to all Hadoop services. I have verified HBase authentication using NIFI. I need to test Kafka as well. In Kafka console, I am able to create a topic. But while producing and consuming I am facing an error like below.

security-protocol is not a recognized option

Below is the Kafka producer command I have tried

/usr/hdp/3.1.0.0-78/kafka/bin/kafka-console-producer.sh --broker-list HOSTNAME:6667 --topic test_new_topic --security-protocol SASL_PLAINTEXT
Giorgos Myrianthous
  • 36,235
  • 20
  • 134
  • 156
pujara
  • 21
  • 1
  • 8

2 Answers2

2

My advise would be to add all the properties under a single file (e.g. client-ssl.properties) with the following content: 

security.protocol=SASL_PLAINTEXT

and finally use –-producer.config to pass the property file to the console producer:

/usr/hdp/3.1.0.0-78/kafka/bin/kafka-console-producer.sh –-broker-list HOSTNAME:6667 –-topic test_new_topic –-producer.config client-ssl.properties

If you don't want to use a property file, you can use --producer-property to pass the security.protocol configuration: 

/usr/hdp/3.1.0.0-78/kafka/bin/kafka-console-producer.sh --broker-list HOSTNAME:6667 --topic test_new_topic  --producer-property security.protocol=SASL_PLAINTEXT
Giorgos Myrianthous
  • 36,235
  • 20
  • 134
  • 156
0

Use --property with the property names as following : 

/usr/hdp/3.1.0.0-78/kafka/bin/kafka-console-producer.sh --broker-list HOSTNAME:6667 --topic test_new_topic  --property security.protocol=SASL_PLAINTEXT
Nishu Tayal
  • 20,106
  • 8
  • 49
  • 101
  • I have used above command now I am facing below warnings [2019-07-10 10:54:56,408] WARN The configuration 'security-protocol' was supplied but isn't a known config. (org.apache.kafka.clients.producer.ProducerConfig) >kakajsj [2019-07-10 10:55:07,409] WARN [Producer clientId=console-producer] Bootstrap broker 192.168.1.182:6667 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient) – pujara Jul 10 '19 at 05:26
  • Kafka is up and running and there is no error in the server logs – pujara Jul 10 '19 at 05:28
  • what kafka version are you using? – Nishu Tayal Jul 10 '19 at 07:16
  • Kafka version 2.0.0 – pujara Jul 10 '19 at 07:23
  • I am running single-node Hadoop (HDP v3.1.0) In the same host-only I am trying to publish and consuming the data – pujara Jul 10 '19 at 13:07
  • For the above issue, I manually edit in zookeeper client using below command set /brokers/ids/1001 {"listener_security_protocol_map":{"SASL_PLAINTEXT":"SASL_PLAINTEXT"},"endpoints":["SASL_PLAINTEXT://HOSTNAME:6667"],"jmx_port":-1,"host":HOSTNAME,"timestamp":"1562767293105","port":6667,"version":4} but here i am facing this error Authentication is not valid : /brokers/ids/1001 – pujara Jul 11 '19 at 05:22
  • Now I am trying to consume the data. I am using below command /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --bootstrap-server HOSTNAME:6667 --topic test --from-beginning --consumer-property security.protocol SASL_PLAINTEXT I am getting below error Caused by: java.lang.IllegalArgumentException: No enum constant org.apache.kafka.common.security.auth.SecurityProtocol. – pujara Jul 12 '19 at 05:46