12

The REST API for Kafka Connect is not secured and authenticated. Since its not authenticated, the configuration for a connector or Tasks are easily accessible by anyone. Since these configurations may contain about how to access the Source System [in case of SourceConnector] and destination system [in case of SinkConnector], Is there a standard way to restrict access to these APIs?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Prabha
  • 121
  • 1
  • 3

4 Answers4

11

In Kafka 2.1.0, there is possibility to configure http basic authentication for REST interface of Kafka Connect without writing any custom code.

This became real due to implementation of REST extensions mechanism (see KIP-285).

Shortly, configuration procedure as follows:

  1. Add extension class to worker configuration file:
rest.extension.classes = org.apache.kafka.connect.rest.basic.auth.extension.BasicAuthSecurityRestExtension
  1. Create JAAS config file (i.e. connect_jaas.conf) for application name 'KafkaConnect':
KafkaConnect {
   org.apache.kafka.connect.rest.basic.auth.extension.PropertyFileLoginModule required
             file="/your/path/rest-credentials.properties";
};
  1. Create rest-credentials.properties file in above-mentioned directory:
user=password
  1. Finally, inform java about you JAAS config file, for example, by adding command-line property to java:
-Djava.security.auth.login.config=/your/path/connect_jaas.conf

After restarting Kafka Connect, you will be unable to use REST API without basic authentication.

Please keep in mind that used classes are rather examples than production-ready features.

Links:

Eugene
  • 2,336
  • 21
  • 28
  • Eugene these instructions are not clear to me if running Kafka Connect as a container. How to build the JAAS config file and command line property (2 and 4) into docker-compose? – DB140141 Feb 26 '20 at 14:49
  • This basic auth is supported since Kafka 2.0.0, see https://issues.apache.org/jira/browse/KAFKA-6776 – echo May 22 '20 at 19:20
  • how to do the step 4 exactly? in some sources they use it as "export KAFKA_OPTS=" some of them don't have the "export". – Bünyamin Şentürk Nov 24 '21 at 12:02
  • you can inject the parameters through KAFKA_JMX_OPTS env var. It has a default value consisting in "-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false" and you can merge it with what you need (what is on step 4), see here: https://github.com/confluentinc/kafka-images/blob/master/kafka-connect-base/include/etc/confluent/docker/launch#L18-L21 – matheuzzy Jan 16 '23 at 16:29
2

This is a known area in need of improvement in the future but for now you should use a firewall on the Kafka Connect machines and either an API Management tool (Apigee, etc) or a Reverse proxy (haproxy, nginx, etc.) to ensure that HTTPS is terminated at an endpoint that you can configure access control rules on and then have the firewall only accept connections from the secure proxy. With some products the firewall, access control, and SSL/TLS termination functions can be all done in a fewer number of products.

Hans Jespersen
  • 8,024
  • 1
  • 24
  • 31
1

As of Kafka 1.1.0, you can set up SSL and SSL client authentication for the Kafka Connect REST API. See KIP-208 for the details.

Gunnar
  • 18,095
  • 1
  • 53
  • 73
  • would please look into my question https://stackoverflow.com/q/55220602/2056178 ? It is right about impossibility of configuring SSL on Connect REST API – Eugene Mar 25 '19 at 08:31
1

Now you are able to enable certificate based authentication for client access to the REST API of Kafka Connect. An example here https://github.com/sudar-path/kc-rest-mtls

sudar-path
  • 11
  • 3