5

The Kafka guide from Quarkus works nicely when running Kafka locally in Docker. I'm trying to change this sample by replacing the local Kafka service with a hosted Kafka service in the cloud which requires TLS.

Does anyone know how I can configure this? In the Quarkus documentation and the Smallrye documentation I don't see any properties for this.

I'd like to use the Kafka service in the IBM Cloud. Based on the documentation I've tried the following configuration in application.properties:

kafka.bootstrap.servers=broker-0-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-4-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-3-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-5-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-2-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-1-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093
kafka.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="...";
kafka.sasl.mechanism=PLAIN
kafka.security.protocol=SASL_SSL
kafka.ssl.protocol=TLSv1.2

Update: I've also tried Gunnar's suggestion below, but it doesn't work. When I use the following application.properties ...

mp.messaging.outgoing.generated-price.connector=smallrye-kafka
mp.messaging.outgoing.generated-price.topic=prices
mp.messaging.outgoing.generated-price.value.serializer=org.apache.kafka.common.serialization.IntegerSerializer

mp.messaging.outgoing.generated-price.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="...";
mp.messaging.outgoing.generated-price.sasl.mechanism=PLAIN
mp.messaging.outgoing.generated-price.security.protocol=SASL_SSL
mp.messaging.outgoing.generated-price.ssl.protocol=TLSv1.2

mp.messaging.incoming.prices.connector=smallrye-kafka
mp.messaging.incoming.prices.topic=prices
mp.messaging.incoming.prices.value.deserializer=org.apache.kafka.common.serialization.IntegerDeserializer

mp.messaging.outgoing.prices.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="...";
mp.messaging.outgoing.prices.sasl.mechanism=PLAIN
mp.messaging.outgoing.prices.security.protocol=SASL_SSL
mp.messaging.outgoing.prices.ssl.protocol=TLSv1.2

kafka.bootstrap.servers=broker-0-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-4-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-3-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-5-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-2-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-1-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093

... I get an error: javax.enterprise.inject.spi.DeploymentException: java.lang.IllegalArgumentException: Invalid channel configuration - the connector attribute must be set for channel prices at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle.onApplicationStart(SmallRyeReactiveMessagingLifecycle.java:22)

Is TLS currently possible for Kafka in Quarkus?

Thanks

Community
  • 1
  • 1
Niklas Heidloff
  • 952
  • 6
  • 13

2 Answers2

4

Have you tried specifying the relevant properties at the channel level? E.g. 

mp.messaging.outgoing.generated-price.connector=smallrye-kafka
mp.messaging.outgoing.generated-price.topic=mytopic
mp.messaging.outgoing.generated-price.ssl.protocol=...
mp.messaging.outgoing.generated-price.ssl.keystore.location=...
mp.messaging.outgoing.generated-price.ssl.keystore.password=...

You also could refer to variables when requiring the same values for multiple topics.

Gunnar
  • 18,095
  • 1
  • 53
  • 73
  • Thanks a lot, Gunnar. Unfortunately this doesn't work for me. Now it complains about a missing connector attribute, even though I haven't touched that one. Do you have any other ideas? – Niklas Heidloff Nov 21 '19 at 14:51
  • You are mixing `incoming` and `outgoing` for the `prices` channel. It should be either one. – Gunnar Nov 21 '19 at 15:08
  • 2
    Aaah. Copy and paste error. It works now! Thanks a lot, Gunnar! – Niklas Heidloff Nov 21 '19 at 15:53
  • @Gunnar, I have a similar issue where Im trying to connect my quarkus application with kafka producer to a broker secured using keycloak. I have the "auth-server-url", "realm", "client-id" and "client_secret", but unable to get this working. Hope you could throw some light on this as I could not find the corresponding application properties reference. – user1057653 Oct 18 '20 at 13:51
0

One property is incorrect in the accepted answer by @Gunnar. It should be "security" instead of "ssl" in the property name.

mp.messaging.outgoing.generated-price.security.protocol=SSL
ashutosh5111
  • 1
  • 3