1

I am using Kafka and facing a problem.

Kafka doesn't seem to work locally using minikube. I get this erorr: org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 1 larger than available brokers: 0.

Kafka config:

apiVersion: apps/v1
kind: Deployment
metadata:
    name: kafka-depl
spec:
    replicas: 1
    selector:
        matchLabels:
            app: kafka
    template:
        metadata:
            labels:
                app: kafka
        spec:
            containers:
                - name: kafka
                  image: bitnami/kafka
                  imagePullPolicy: Never
                  env:
                      - name: KAFKA_BROKER_ID
                        value: '0'
                      - name: KAFKA_ZOOKEEPER_CONNECT
                        value: zookeeper-srv:2181
                      - name: KAFKA_ADVERTISED_LISTENERS
                        value: PLAINTEXT://kafka-srv:9092
                      - name: ALLOW_PLAINTEXT_LISTENER
                        value: 'yes'

---
apiVersion: v1
kind: Service
metadata:
    name: kafka-srv
spec:
    selector:
        app: kafka
    ports:
        - name: kafka
          protocol: TCP
          port: 9092
          targetPort: 9092

Zookeeper config:

apiVersion: apps/v1
kind: Deployment
metadata:
    name: zookeeper-depl
spec:
    replicas: 1
    selector:
        matchLabels:
            app: zookeeper
    template:
        metadata:
            labels:
                app: zookeeper
        spec:
            containers:
                - name: zookeeper
                  image: bitnami/zookeeper:latest
                  imagePullPolicy: Never
                  env:
                      - name: ALLOW_ANONYMOUS_LOGIN
                        value: 'yes'
---
apiVersion: v1
kind: Service
metadata:
    name: zookeeper-srv
spec:
    selector:
        app: zookeeper
    ports:
        - name: zookeeper
          protocol: TCP
          port: 2181
          targetPort: 2181

I have deployed these on Azure and it's working correctly and locally using docker-compose.

However, running it locally with minikube doesn't work.

Kafka pod logs

Zookeeper pod logs

Adham Salama
  • 143
  • 1
  • 9
  • I'd strongly recommend using an operator such as Strimzi, not a manual deployment based from Docker Compose. I've found that doesn't setup networking correctly. That also comes with a KafkaTopic CRD – OneCricketeer May 01 '22 at 05:00
  • You may refer this for troubleshooting :https://stackoverflow.com/questions/33098366/error-creating-kafka-topics-replication-factor-larger-than-available-brokers – RahulKumarShaw May 02 '22 at 08:10
  • does this above links works for you? – RahulKumarShaw May 03 '22 at 06:17

1 Answers1

-1

Seems like your server not started , check logs for issue

Kill process

sudo fuser -k 2181/tcp

run zookeeper

bin/zookeeper-server-start.sh config/zookeeper.properties

Run Kafka

bin/kafka-server-start.sh config/server.properties

You can check this thread which is related to your issue, You can also follow the others answer for the troubleshooting purpose.

RahulKumarShaw
  • 4,192
  • 2
  • 5
  • 11