0

I am trying to use this method of deploying Kafka to Openshift: https://github.com/mattf/openshift-kafka

The deployment sets up and I can set up a broker and consumer inside the container and they work fine. When I tried to access a topic using a console-consumer (using port-forward to my local machine) i got a message saying that a broker is not available.

I tried adding advertised.listeners=PLAINTEXT://openshiftRoute:80 override to the yaml file but now I am getting Error while fetching metadata with correlation id X : {test=LEADER_NOT_AVAILABLE}.

When I try using the Openshift route to the localhost:9092

kafka-console-consumer.bat --bootstrap-server http://route:80 --topic test --from-beginning

i get 

[2019-05-14 21:58:47,773] ERROR Error processing message, terminating consumer process:  (kafka.tools.ConsoleConsumer$)
java.lang.OutOfMemoryError: Java heap space

What am I missing here, how can I access Kafka externally in Openshift?

Matt
  • 556
  • 8
  • 31

1 Answers1

0

If you want to simplify the way to do that, you could try the Strimzi project (https://strimzi.io) for deploying and managing an Apache Kafka cluster on Kubernetes and OpenShift. It provides a really simple way for exposing the Kafka cluster outside of OpenShift using routes (but even supporting load balancers and node ports). You can read more here: https://strimzi.io/docs/latest/#con-kafka-listeners-deployment-configuration-kafka

ppatierno
  • 9,431
  • 1
  • 30
  • 45
  • Actually I did try to use strimzi at my first attempt but all I got when trying to use the openshift route to access kafka was broker not available. – Matt May 14 '19 at 20:18
  • Let's talk about that, you can join the related slack channel or doing here. What's your YAML file describing the Kafka cluster? – ppatierno May 14 '19 at 20:20
  • https://stackoverflow.com/questions/56083914/setting-up-kafka-on-openshift-with-strimzi Here is a question I asked about using hte strimzi method, maybe you could look at it and see if there is anything you see I am doing wrong? All the yamls I used while trying stimzi were the ones given in the redhat blog that I linked in that question. – Matt May 14 '19 at 20:23
  • Just answered to that post – ppatierno May 14 '19 at 20:28