3

We are currently running an unsecured Kafka setup on AWS MSK (so I don't have access to most config files directly and need to use the kafka-cli) and are looking into ways to add protection. Setting up TLS & SASL is easy, though as our Kafka cluster is behind a VPN and already has restricted access does not add more security.

We want to start with the most important and in our opinion quick win security addition. Protect topics from being deleted (and created) by all users. We currently have allow.everyone.if.no.acl.found set to true.

All I find on Google or Stack Overflow shows me how I can restrict users from reading/writing to other topics than they have access to. Though Ideally that is not what we want to implement as a first step.

I have found things about a root-user (Is an admin user, though was called root in all tutorials I read). Though the examples I have found don't show examples of adding an ACL to this root user to make it the only one accessible, the topic deletion/creation.

Can you please explain how to create a user that, and block all other users?

By the way, we also don't use zookeeper, even though an MSK-cluster ads this per default. And hope we can do this without adding zookeeper actively to our stack. The answer given here hardly relies on zookeeper. Also, this answer points to the topic read/write examples only, even though the question was the same as I am asking

Dennis van de Hoef - Xiotin
  • 1,481
  • 1
  • 13
  • 29
  • Why can't you use IAM roles? You can allow user specific cluster & topic to read from. – Snigdhajyoti Jan 04 '22 at 20:10
  • That is a valid question, I can't use IAM roles as the Ruby library depends on librdkafka, and they [don't want to add support](https://github.com/edenhill/librdkafka/issues/3402) for the needed mechanism AWS_MSK_IAM that we would need. IAM would have been my preferred way, though not with this added complexity. – Dennis van de Hoef - Xiotin Jan 05 '22 at 07:16

2 Answers2

3

I'd like to start with a disclaimer that I'm personally not familiar with AWS MSK offering in great detail so this answer is largely based on my understanding of the open source distribution of Apache Kafka.

First - The Kafka ACLs are actually stored in Zookeeper by default so if you're not using Zookeeper, it might be worth adding this if you're not using it.

Reference - Kafka Definitive Guide - 2nd edition - Chapter 11 - Securing Kafka - Page 294

Second - If you're using SASL for authentication through any of the supported mechanisms such as GSSAPI (Kerberos), then you'll need to create a principal as you would normally create one and use one of the following options:

  1. Add the required permissions for topic creation/deletion etc. using the kafka-acls command (Command Reference)

    bin/kafka-acls.sh --add --cluster --operation Create --authorizer-properties zookeeper.connect=localhost:2181 --allow-principal User:admin

    Note - admin is the assumed principal name

  2. Or add admin user to the super users list in server.properties file by adding the following line so it has unrestricted access on all resources

    super.users=User:Admin

    Any more users can be added in the same line delimited by ;.

To add the strictness, you'll need to set allow.everyone.if.no.acl.found to false so any access to any resources is only granted by explicitly adding these permissions.

Third - As you've asked specifically about your root user, I'm assuming you're referring to the linux root here. You could just restrict the linux level permissions using chmod command for the kafka-acls.sh script but that is quite a crude way of achieving what you need. I'm also not entirely sure if this is doable in MSK or not.

Lalit
  • 1,944
  • 12
  • 20
  • Thanks for your response. The root user is the admin user. Though in all documentation I read it was called the root user, so that is why I used that name :) Even though your answer is fantastic explaining. It does not help me with my question|: setting access to delete/create a topic to a certain user without adding access rights to all other users. Can you maybe add something that answers my question or helps me on my search? – Dennis van de Hoef - Xiotin Dec 20 '21 at 06:03
  • You will get this with the steps mentioned. You have already denied everything to everyone else... After the steps suggested, you will have a specific admin user with create permissions and no one else... yiu would need to add access for specific users as and when needed. – Lalit Dec 20 '21 at 09:48
  • Or did I not get the question? – Lalit Dec 20 '21 at 09:49
  • You got the question correct, I just was not understanding it. The comment you added here made me understand it. Thanks! – Dennis van de Hoef - Xiotin Dec 21 '21 at 06:22
  • Oh ok. Let me add that part as well in the answer then. The answer should be good enough for the reader to understand. :)Thanks for the feedback. :) – Lalit Dec 21 '21 at 10:07
  • @Lalit, We have added ACL like below to restrict topic creation " bin/kafka-acls.sh --add --cluster --operation Create --authorizer-properties zookeeper.connect=localhost:2181 --allow-principal User:myadminusername" And still Im able to create topics without specifying any user details. And We've set allow.everyone.if.no.acl.found=false Can you suggest how to limit topic creation. – Rajashekhar Meesala Jul 18 '22 at 11:49
  • @RajashekharMeesala - Apologies for the delay in my response. Would you be able to share the details of your situation in a separate question? Because in such security scenarios, very small differences can cause major changes. I'll be happy to investigate your situation with you. – Lalit Aug 18 '22 at 13:57
0

For the record, I ran the following commands to make it work.

1 admin (root) user that can manage the cluster, and despite allow.everyone.if.no.acl.found set to true all users without an ACL can only read/write to topics.

# Define ACL for test_admin user
/kafka/bin/kafka-acls.sh \
  --bootstrap-server $BROKERS \
  --command-config /app/accounts/test_admin.properties \
  --add \
  --allow-principal User:test_admin \
  --operation Alter \
  --operation Create \
  --operation Describe \
  --operation DescribeConfigs \
  --allow-host '*' \
  --cluster

# Define ACL for Broker (ANONYMOUS) user
/kafka/bin/kafka-acls.sh \
  --bootstrap-server $BROKERS \
  --command-config /app/accounts/test_admin.properties \
  --add \
  --allow-principal User:ANONYMOUS \
  --operation ALL \
  --cluster

# Define ACL for all users to only give them read/write
/kafka/bin/kafka-acls.sh \
  --bootstrap-server $BROKERS \
  --command-config /app/accounts/test_admin.properties \
  --add \
  --allow-principal User:* \
  --operation Read \
  --operation Write \
  --topic '*' \
  --allow-host '*'
Dennis van de Hoef - Xiotin
  • 1,481
  • 1
  • 13
  • 29