Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [kafka]

Kafka is a distributed messaging queue application written in java.

Kafka website: http://kafka.apache.org/

When asking a question about provides relevant information like the kafka version and the version. The design of your cluster should be mentioned with the number of brokers, the topic configuration.

When you have a bug on a particular topic, perform the same test on another topic with different parameters and observe if you can reproduce.

97 questions
0
votes
1 answer

Kafka + Query the detailed configuration of all Topics

we have Kafka servers cluster with Kafka version 2.6 on Linux RHEL machines machine in the cluster are - kafka1,kafka2,kafka3 usually we are use the following cli in order to get the detailed configuration of all Topics kafka-topics.sh --zookeeper…
King David
  • 549
  • 6
  • 20
0
votes
1 answer

Kafka Broker Runs locally via docker compose but fails in kubernetes with near identical config

Kafka broker runs beautifully when running locally via docker-compose. version: "3" services: zookeeper: image: 'bitnami/zookeeper:latest' ports: - '2181:2181' environment: - ALLOW_ANONYMOUS_LOGIN=yes kafka: image:…
Zevrant
  • 11
  • 2
0
votes
1 answer

How to manage storage / persistence overflow in Bitnami Kafka Helm chart?

I deploy Bitanmi Kafka Helm chart on AWS provisioned with Terraform. I find the documentation storage and persistence allocation, very confusing. From what I understood from the documentation; logs are chunks of messages in a topic, when…
Rubber Duck
  • 101
  • 2
0
votes
1 answer

How to rotate compressed log file on centos 7?

I have customized kafka server and the app is creating log files and configures log rotation. My problem is that I need to compress the rotation logs that were created. I tried to create new log rotation file on /etc/logrotate.d with the following…
Rasa
  • 1
0
votes
1 answer

How to implement a secured kafka environment with a public interface?

Scenario: Kafka cluster, secured with SSL encryption and SASL/PLAIN authentication, resides in an AWS private subnet within a dedicated VPC. Within the private subnet, all is fine. I use self-generated CA and keys to secure the communication, host…
mjahr
  • 3
  • 4
0
votes
1 answer

How set a fixed private IP for ECS containers?

I have a Task Definition called kafka-zookeeper, it has an Apache Kafka broker and a Zookeeper node in it. Those containers need to communicate with each other (that's easy) and with the kafka and zookeeper containers from other kafka-zookeeper…
ItsaMeTuni
  • 161
  • 1
  • 5
0
votes
1 answer

Filebeat kafka input with SASL?

I'm trying to get filebeat to consume messages from kafka using the kafka input. I'm unable to authenticate with SASL for some reason and I'm not sure why that is. The documentation for both Kafka and Filebeat is a little lacking when trying to use…
vane
  • 155
  • 1
  • 2
  • 9
1 2 3 4 5 6
7