I have a local k3s Kubernetes cluster created by multipass.
I am trying to set up Kafka using Ingress way based on this tutorial to make it accessible for clients running outside of Kubernetes.
Here are my steps:
First get my cluster IP by
➜ kubectl get nodes
NAME STATUS ROLES AGE VERSION
west-master Ready control-plane,master 15m v1.26.3+k3s1
➜ kubectl get node west-master -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
west-master Ready control-plane,master 16m v1.26.3+k3s1 192.168.205.5 <none> Ubuntu 22.04.2 LTS 5.15.0-67-generic containerd://1.6.19-k3s1
➜ kubectl cluster-info
Kubernetes control plane is running at https://192.168.205.5:6443
CoreDNS is running at https://192.168.205.5:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://192.168.205.5:6443/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy
which is 192.168.205.5 (Is this correct IP address for the one I used in my-kafka-persistent.yaml I used in next step?).
Then I deploy my Kafka by:
kubectl create namespace hm-kafka
kubectl apply --filename="https://strimzi.io/install/latest?namespace=hm-kafka" --namespace=hm-kafka
kubectl apply --filename=my-kafka-persistent.yaml --namespace=hm-kafka
my-kafka-persistent.yaml (based on kafka-persistent.yaml):
---
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: hm-kafka
spec:
kafka:
version: 3.4.0
replicas: 3
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: tls
port: 9093
type: internal
tls: true
- name: external
port: 9094
type: ingress
tls: true
configuration:
bootstrap:
host: kafka-bootstrap.192.168.205.5.nip.io
brokers:
- broker: 0
host: kafka-broker-0.192.168.205.5.nip.io
- broker: 1
host: kafka-broker-1.192.168.205.5.nip.io
- broker: 2
host: kafka-broker-2.192.168.205.5.nip.io
config:
offsets.topic.replication.factor: 3
transaction.state.log.replication.factor: 3
transaction.state.log.min.isr: 2
default.replication.factor: 3
min.insync.replicas: 2
inter.broker.protocol.version: "3.4"
storage:
type: jbod
volumes:
- id: 0
type: persistent-claim
size: 100Gi
deleteClaim: false
zookeeper:
replicas: 3
storage:
type: persistent-claim
size: 100Gi
deleteClaim: false
entityOperator:
topicOperator: {}
userOperator: {}
After deployment:
Pods
Services
Ingresses
Also, for each Ingress, I can see SSL passthrough in the annotations:
Then I succeed following the tutorial creating the truststore.
➜ kubectl get secret hm-kafka-cluster-ca-cert \
--namespace=hm-kafka \
--output=jsonpath="{.data.ca\.crt}" \
| base64 -d \
> ca.crt
➜ keytool -importcert \
-trustcacerts \
-alias root \
-file ca.crt \
-keystore kafka-truststore.jks \
-storepass my_passw0rd \
-noprompt
Certificate was added to keystore
However, when I try to produce data, I met this issue:
➜ kafka-console-producer \
--broker-list kafka-bootstrap.192.168.205.5.nip.io:443 \
--producer-property security.protocol=SSL \
--producer-property ssl.truststore.password=my_passw0rd \
--producer-property ssl.truststore.location=kafka-truststore.jks \
--topic my-topic
>[2023-04-14 15:57:06,047] ERROR [Producer clientId=console-producer] Connection to node -1 (kafka-bootstrap.192.168.205.5.nip.io/192.168.205.5:443) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2023-04-14 15:57:06,047] WARN [Producer clientId=console-producer] Bootstrap broker kafka-bootstrap.192.168.205.5.nip.io:443 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2023-04-14 15:57:06,200] ERROR [Producer clientId=console-producer] Connection to node -1 (kafka-bootstrap.192.168.205.5.nip.io/192.168.205.5:443) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2023-04-14 15:57:06,201] WARN [Producer clientId=console-producer] Bootstrap broker kafka-bootstrap.192.168.205.5.nip.io:443 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2023-04-14 15:57:06,691] ERROR [Producer clientId=console-producer] Connection to node -1 (kafka-bootstrap.192.168.205.5.nip.io/192.168.205.5:443) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2023-04-14 15:57:06,691] WARN [Producer clientId=console-producer] Bootstrap broker kafka-bootstrap.192.168.205.5.nip.io:443 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
Any guide would be appreciate, thanks!
UPDATE 1
Thanks @OneCricketeer pointing the issue!
As I am using multipass on macOS, I can provide INSTALL_K3S_EXEC="server --disable traefik", so the updated command to create k3s cluster is:
multipass launch --name=west-master --cpus=4 --memory=16g --disk=128g
multipass exec west-master -- \
bash -c 'curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --disable traefik" K3S_KUBECONFIG_MODE="644" sh -'
I actually switched to Rancher Desktop as it is also using k3s and easy to disable Traefik which can be set up in the UI.
UPDATE 2
Regarding how to deploy ingress-nginx and how to resolve another issue "ingress does not contain a valid IngressClass" I met, I posted at Strimzi Kafka brokers not be created because of "ingress does not contain a valid IngressClass"
