Stack Overflow
Stack Overflow

Questions tagged [apache-kafka-security]

The following security measures are currently supported:

  1. Authentication of connections to brokers from clients (producers and consumers), other brokers and tools, using either SSL or SASL. Kafka supports the following SASL mechanisms:
    • SASL/GSSAPI (Kerberos) - starting at version 0.9.0.0
    • SASL/PLAIN - starting at version 0.10.0.0
    • SASL/SCRAM-SHA-256 and SASL/SCRAM-SHA-512 - starting at version 0.10.2.0
    • SASL/OAUTHBEARER - starting at version 2.0
  2. Authentication of connections from brokers to ZooKeeper
  3. Encryption of data transferred between brokers and clients, between brokers, or between brokers and tools using SSL (Note that there is a performance degradation when SSL is enabled, the magnitude of which depends on the CPU type and the JVM implementation.)
  4. Authorization of read / write operations by clients
  5. Authorization is pluggable and integration with external authorization services is supported

Links

Related Tags

55 questions
2
votes
1 answer

Is there a way to set up Kafka ACL to allow using any consumer group without listing them

I am trying to set up Kafka, where each user have several topics, but each topic may be consumed with any number of consumer group by the user the topic belongs to. Kafka server version used: kafka_2.12-2.4.0 (Commit:77a89fcf8d7fa018) Kafka client…
Skrog
  • 21
  • 1
  • 1
  • 4
2
votes
2 answers

Kafka Schema Registry - Error starting the schema registry (io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication)

i'm trying to configure schema registry to work with SSL, i have already zookeeper and kafka brokers working with the same SSL keys. but whenever i start the schema-registry i get the following error ERROR Error starting the schema registry …
LoayN
  • 31
  • 1
  • 5
2
votes
0 answers

Kafka Failed to create new KafkaAdminClient on Kerberized Cluster

i have a kerberized cluster with Kafka on it. I want to use Confluent Schema Registry with Kafka on cluster. Launching the Schema Registry from my local pc, everything works just fine. But when i uploaded it on a machine in the cluster and tried to…
Lorenzo Di Giacomo
  • 129
  • 3
  • 13
2
votes
0 answers

Adding truststore certificate to Apache Kafka without bouncing Kafka cluster

We are using Apache kafka_2.11-1.1.0. Whenever we want to add a new Kafka producer or consumer, we update the new certificate in truststore. Once the cert is added, we have to restart the kafka cluster. Is there a way to add cert without Rolling…
c0der512
  • 546
  • 5
  • 18
2
votes
2 answers

Setting Python KafkaProducer sasl mechanism property

The sasl mechanism we are using is SCRAM-SHA-256 but the kafka producer will only accept sasl_mechanism as PLAIN, GSSAPI, OAUTHBEARER The following config will give the error sasl_mechanism must be in PLAIN, GSSAPI, OAUTHBEARER config …
AnonymousAlias
  • 1,149
  • 2
  • 27
  • 68
2
votes
1 answer

Kstreams how to restrict creating intermediate topics

Is there a way can a operations team restrict application teams from creating kafka stream intermediate topics on kafka cluster?
sanat
  • 91
  • 7
1
vote
0 answers

Is it possible to sync secrets between Azure Key Vault and Kafka on HDinsight?

I currently have an application deployed on AKS, which produces to a Kafka topic, with Kafka deployed on HDinsight. I want to implement a SASL/OAUTHBEARER as the security mechanism. However, I'd also like the secrets to be stored in Azure Key Vault…
Houssen Jivan
  • 11
  • 1
1
vote
1 answer

Manage Kafka ACLs from C#

I have an application that communicates with Kafka using the C# (.NET 6) Confluent.Kafka library. It's part of a larger platform that's mostly written in Java. On the Java side org.apache.kafka.AdminClient provides ACLOperations classes to manage…
Jason
  • 121
  • 8
1
vote
2 answers

is it possible to refer environmental variable as values for properties in the kafkaProducer.properties and KafkaConsumer.properties files of Kafka

Im have a producer and consumer java code and Im trying to upgrade it to connect with the Kafka which is secured with SSL. I'm in a situation that the ssl related passwords should be given only via environmental variables. So is it possible to…
rajesh
  • 179
  • 1
  • 3
  • 16
1
vote
0 answers

How to make Kafka security setup SASL_SSL with oAuth on Windows box?

I have enabled SASL_SSL configuraion , And tested with clients producing & consuming…
user6006387
1
vote
0 answers

Is it possible to configuring Kafka SASL authentication on the osquery agent?

I'm trying setup osquery logging as a Kafka producer w/ SASL-plain authentication. But not sure is it possible or not. According to this page https://osquery.readthedocs.io/en/stable/deployment/logging/#logging-as-a-kafka-producer There are 3 Kafka…
user6060858
  • 11
  • 1
1
vote
1 answer

Schema Registry security configuration

Tried to configure Schema registry with SSL. When attempted to use HTTPS schema registry with AVRO Producer, it failed with error No SAN Name found I had configured to ignore the host name checking. but still didnt work. Similar configuration had…
WebDeveloper2017
  • 73
  • 1
  • 9
1
vote
1 answer

Failure to re-authenticate Kafka client SASL connection, should close connection

As per KIP-368 (https://cwiki.apache.org/confluence/display/KAFKA/KIP-368), when 'connections.max.reauth.ms' is explicitly set to a positive number the server will disconnect any SASL connection that does not re-authenticate. If the…
KafkaNoob
  • 21
  • 3
1
vote
2 answers

How to enable security in a Kafka cluster without having downtime

We have a Kafka cluster in production without any security. We plan to turn on security (SASL/OAUTHBEARER) on the broker side. But looks like as soon as we turn on broker side security all the insecure client will be dropped immediately. For smooth…
KafkaNoob
  • 21
  • 3
1
vote
1 answer

how to give topic access to one specific user?

I am collecting the data from different resources, each resource has one specific topic for each client. I want to give the access for each user only to the corresponding topic, so they can't have access to all the topics. I am working with Kafka…
Aymen Rahal
  • 53
  • 1
  • 7
1
2
3 4