kcat is a single-binary Apache Kafka CLI tool
kcat is a single-binary Apache Kafka CLI tool for doing Unix cat-like operations against topics, such as reading (consuming), writing (producing), as well as Kafka related network troubleshooting.
Questions tagged [kcat]
49 questions
0
votes
0 answers
kcat throwing error on Manjaro Linux with kafka
kcat version on Manjaro/KDE:
kcat -V
kcat - Apache Kafka producer and consumer tool
https://github.com/edenhill/kcat
Copyright (c) 2014-2021, Magnus Edenhill
Version 1.7.0 (JSON, Avro, Transactions, IncrementalAssign, librdkafka 1.8.2…
chovy
- 72,281
- 52
- 227
- 295
0
votes
1 answer
How to manipulate offsets of the source database for Debezium
So I've been experimenting with Kafka and I am trying to manipulate/change the offsets of the source database using this link https://debezium.io/documentation/faq/. I was successfully able to do it but I was wondering how I would do this in native…
Alex
- 211
- 1
- 11
0
votes
1 answer
Bitnami - exec: "zookeeper-shell.sh": executable file not found in $PATH: unknown
I have installed kafka on docker with following yml
version: "3"
services:
zookeeper:
image: 'bitnami/zookeeper:latest'
ports:
- '2181:2181'
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
kafka:
image: 'bitnami/kafka:latest'
ports:
- '9092:9092'
…
murmansk
- 845
- 2
- 11
- 28
0
votes
1 answer
Cannot read data from Kafka partition on exact listener port
I have Kafka cluster of 3 nodes. I am using kafkacat to list data from Kafka. I configure PLAINTEXT and VPN_PLAINTEXT…
dorinand
- 1,397
- 1
- 24
- 49
0
votes
2 answers
`exec.Command()` produces no output when running `kcat`
I need to wrap kcat in a Go function to read a series of topic messages, so thought to use exec.Command() for this is as follows:
package main
import (
"fmt"
"os/exec"
)
func main() {
cmd := exec.Command("kcat", "-b…
Traiano Welcome
- 783
- 2
- 12
- 24
0
votes
1 answer
How to load data from json file into kafka
I have a mydata.json file that is like below:
{
"student_id": "Student 01",
"grades": [
{
"english": "A",
"semester": 1
},
{
"math": "B",
"semester": 2
}
],
"attendance": [
{
"present":…
Anthony
- 33,838
- 42
- 169
- 278
0
votes
0 answers
Syntax for kafkacat to consume specified range of records
This kcat repo has an example of how to read the last N (=2000) records in a Kafka topic (syslog in this case):
$ kcat -C -b mybroker -t syslog -p 0 -o -2000 -e
The parameter -o is offset from the end of the topic, and -e indicates till the end of…
Tristan Tran
- 1,351
- 1
- 10
- 36
0
votes
1 answer
Kerberros GSSAPI doesn't work within kafkacat alpine container
Previously I've reported it into kafkacat tracker but the issue has been closed as related to cyrus-sasl/krb5.
podman run --rm -it --name kafkacat-DEV \
-v$(pwd)/conf/integration:/conf -v$(pwd)/conf/integration/krb5.conf:/etc/krb5.conf \
…
Hubbitus
- 5,161
- 3
- 41
- 47
0
votes
1 answer
Kafkacat unable to send messages to topics
I'm new to Apache Kafka and Kafkacat trying to learn new things from here: https://www.youtube.com/watch?v=5Mgni6AYnWg&t=123s at time 17:32.
I setup a kafka and kafkacat on my remote VM.
[user@user bin]$ ./kafka-topics.sh --list --zookeeper…
PAA
- 1
- 46
- 174
- 282
0
votes
1 answer
Kafkacat how to republish a binary message maintaining key
I have been trying to use kafkacat to find a message in a topic and publish it back into the topic. We use protobuf so the message values should be in bytes (Keys can be different such as strings or bytes). However, I am unable to publish the…
Thomas Freiling
- 67
- 1
- 8
0
votes
0 answers
Is it Possible to modify or update message of a topic by kafkacat?
I am trying to built an alert streaming application with confluent platform community & ksql headless mode. My application flow is I produce message (json format which have column like threshold, alert_flag, device_id etc.) to a base ksql stream…
Saiful Islam
- 186
- 1
- 3
- 13
0
votes
1 answer
how manually analyse headers from kafka message topics? Getting confluentinc_kafkacat_1 exited with code 2
Goal: I want to analyse the headers from a topic and I am looking for some straighforward way to see the header. So I don't want to develop an extra application or extense code just for that. Any straighforward tool to see the header will be…
Jim C
- 3,957
- 25
- 85
- 162
0
votes
2 answers
Syntax error while using COPY to fill specific columns in Postgres table from stdin
I have a syntax error in my bash command when I try to fill a Postgres table with data from Kafka using kafkacat:
[k ~]$ kafkacat -b XXX.XX.Y.Z:9092 -t test -o 20501 -f '%k|%s|%T|%p|%o|213\n' -e | psql -c "copy raw_from_kafka(key, value, timestamp,…
Konstantin Popov
- 1,546
- 4
- 15
- 19
0
votes
1 answer
Kafka on Kubernetes in minikube not working
I am trying to setup Kafka on Minikube, a very basic setup. I can't validate if Kafka and Zookeeper have been setup correctly because kafkacat fails.
Here is my config:
zookeeper
kind: Deployment
apiVersion: apps/v1
metadata:
name:…
maopuppets
- 420
- 3
- 9
- 27
0
votes
1 answer
Set timestamp using kafkacat when producing
I want to be able to transmit messages to kafka using kafkacat but also set the timestamp of the messages. But it seems like this is not possible. Is this true or is there something that I am missing. I can't seem to find a full kafkacat reference…
Steven
- 69
- 5