kcat throwing error on Manjaro Linux with kafka

Question

kcat version on Manjaro/KDE:

kcat -V
kcat - Apache Kafka producer and consumer tool
https://github.com/edenhill/kcat
Copyright (c) 2014-2021, Magnus Edenhill
Version 1.7.0 (JSON, Avro, Transactions, IncrementalAssign, librdkafka 1.8.2 builtin.features=gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer)

Curl command from shell:

curl -X "POST" "http://localhost:8080/tasks" -v \
    -H "Idempotency-Key: 9417E83E-313E-468E-AC7C-1BCE0BAF9712" \
    -H "Content-Type: application/json" \
    -d "{\"name\":\"Read the docs\"}"

This command produces the following errors:

echo "{\"greeting\":\"Hello, world `date`\"}" | \
    kcat -P \
         -b localhost:9092 \
         -t items-responses \
         -k "5cf7a1d5-3772-49ef-86e7-ba6f2c7d7d07" \
         -H ":status=200" \
         -H "zilla:correlation-id=1-e75a4e507cc0dc66a28f5a9617392fe8"

client side error (from kcat):

%5|1662750342.235|REQTMOUT|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Timed out ApiVersionRequest in flight (after 10005ms, timeout #0)
%4|1662750342.235|FAIL|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiVersionRequest failed: Local: Timed out: probably due to broker version < 0.10 (see api.version.request configuration) (after 10005ms in state APIVERSION_QUERY)
%4|1662750342.235|REQTMOUT|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Timed out 1 in-flight, 0 retry-queued, 0 out-queue, 0 partially-sent requests
% ERROR: Local: Broker transport failure: localhost:9092/bootstrap: ApiVersionRequest failed: Local: Timed out: probably due to broker version < 0.10 (see api.version.request configuration) (after 10005ms in state APIVERSION_QUERY)
% ERROR: Local: All broker connections are down: 1/1 brokers are down: terminating
➜  http.kafka.cache git:(main) ✗

Here is a link to a fully reproducible example:

https://github.com/aklivity/zilla-examples/tree/main/http.kafka.sync

Just follow instructions in README assuming you have docker, curl and kcat-cli installed.

It works fine on macOS

edit: output as requested:

$ kcat -L -b localhost:9092
% ERROR: Failed to acquire metadata: Local: Broker transport failure (Are the brokers reachable? Also try increasing the metadata timeout with -m <timeout>?)

So, you have an HTTP server executing shell kcat commands rather than having an embedded Kafka client library in it? Or, why not `docker-compose exec` to run `kafka-console-producer`inside the container? (I'm not sure that port forward is correct, but I haven't tried Kafka Kraft node) — OneCricketeer, Sep 11 '22 at 02:11
I'm not sure how the curl command is relevant, and haven't heard of Zilla tool. Confluent REST Proxy also offers HTTP endpoints for producers, though — OneCricketeer, Sep 11 '22 at 02:18
You're using that Docker Compose file in the repo? Then that means your port forward isn't working. See [these sections](https://github.com/bitnami/bitnami-docker-kafka#kafka-without-zookeeper-kraft) of the docs and [this blog](https://www.confluent.io/blog/kafka-listeners-explained/) — OneCricketeer, Sep 11 '22 at 17:09

kcat throwing error on Manjaro Linux with kafka

0 Answers0