3

I've been following this blog post to implement an embedded sasl_ssl https://sharebigdata.wordpress.com/2018/01/21/implementing-sasl-plain/

@SpringBootTest
@RunWith(SpringRunner.class)
@TestPropertySource(properties = {
        "spring.kafka.bootstrap-servers=${spring.embedded.kafka.brokers}",
        "spring.kafka.consumer.group-id=notify-integration-test-group-id",
        "spring.kafka.consumer.auto-offset-reset=earliest"
})
public class ListenerIntegrationTest2 {
    static final String INBOUND = "inbound-topic";
    static final String OUTBOUND = "outbound-topic";

    static {
        System.setProperty("java.security.auth.login.config", "src/test/java/configs/kafka/kafka_jaas.conf");
    }

    @ClassRule
    public static final EmbeddedKafkaRule KAFKA = new EmbeddedKafkaRule(1, true, 1,
            ListenerIntegrationTest2.INBOUND, ListenerIntegrationTest2.OUTBOUND)
            .brokerProperty("listeners", "SASL_SSL://localhost:9092, PLAINTEXT://localhost:9093")
            .brokerProperty("ssl.keystore.location", "src/test/java/configs/kafka/kafka.broker1.keystore.jks")
            .brokerProperty("ssl.keystore.password", "pass")
            .brokerProperty("ssl.key.password", "pass")
            .brokerProperty("ssl.client.auth", "required")
            .brokerProperty("ssl.truststore.location", "src/test/java/configs/kafka/kafka.broker1.truststore.jks")
            .brokerProperty("ssl.truststore.password", "pass")
            .brokerProperty("security.inter.broker.protocol", "SASL_SSL")
            .brokerProperty("sasl.enabled.mechanisms", "PLAIN,SASL_SSL")
            .brokerProperty("sasl.mechanism.inter.broker.protocol", "SASL_SSL");

When I use the PLAINTEXT://localhost:9093 config I get the following: WARN org.apache.kafka.clients.NetworkClient - [Controller id=0, targetBrokerId=0] Connection to node 0 terminated during authentication. This may indicate that authentication failed due to invalid credentials.

However, when I remove it, I get org.apache.kafka.common.KafkaException: Tried to check server's port before server was started or checked for port of non-existing protocol

I've tried changing the SecurityProtocol type to autodiscover which style of broker communication it should be using (it's hardcoded to plaintext - this should probably get fixed):

   if (this.kafkaPorts[i] == 0) {
      this.kafkaPorts[i] = TestUtils.boundPort(server, SecurityProperties.forName(this.brokerProperties.getOrDefault("security.protocol", SecurityProtocol.PLAINTEXT).toString()); // or whatever property can give me the security protocol I should be using to communicate
   }

I still get the following error: WARN org.apache.kafka.clients.NetworkClient - [Controller id=0, targetBrokerId=0] Connection to node 0 terminated during authentication. This may indicate that authentication failed due to invalid credentials.

Is there a way to correctly configure embedded kafka to be sasl_ssl enabled?

bryce
  • 479
  • 1
  • 6
  • 13
  • I don't know the answer, but I am intrigued as to why you are trying. The embedded broker is only intended for testing. – Gary Russell Jun 07 '19 at 19:18
  • There are some situations where I have to integrate with a kafka cluster secured with SASL_SSL. As a result, I cannot build a CI/CD app without appropriately testing my client is correctly configured to connect to the cluster in the first place. – bryce Jun 10 '19 at 16:38
  • Assuming you can connect to the real broker OK, I would suggest comparing the `... [main] INFO kafka.server.KafkaConfig - KafkaConfig values: ` emitted for the embedded broker with those from a real broker, to see what's different. – Gary Russell Jun 10 '19 at 17:30
  • From what I can tell, the configs are roughly the same. The issue is coming from the embedded kafka for two reasons: 1.) The embedded kafka is configured to only use PLAINTEXT protocol to communicate to the broker. 2.) Zookeeper options are not configurable from what I can tell in the embedded kafka. I think what is happening is that the underlying client that is creating the initial topics is getting denied from the configured SASL_SSL embedded kafka. – bryce Jun 10 '19 at 20:55
  • It looks like you are using hard-coded ports 9092/9093. But your clients are binding with `"spring.kafka.bootstrap-servers=${spring.embedded.kafka.brokers}",` which will only return the plaintext port, as you have stated. I agree that should be fixed (please open an issue in GitHub). In the meantime, try adding `.kafkaPorts(9092)` to hard-wire the broker list to use your explicit SASL port; that will avoid the lookup there. – Gary Russell Jun 10 '19 at 21:47
  • Did you find any solution for this? I also am facing same issue. – Shades88 May 12 '20 at 11:52

0 Answers0