Stack Overflow
Stack Overflow

Questions tagged [kafka-rest]

Provides a RESTful interface to a Kafka cluster, so you can produce and consume messages, as well as perform administrative actions through any REST client.

Confluent REST Proxy

(From the official Readme.md linked below)

The Kafka REST Proxy provides a RESTful interface to a Kafka cluster. It makes it easy to produce and consume messages, view the state of the cluster, and perform administrative actions without using the native Kafka protocol or clients. Examples of use cases include reporting data to Kafka from any frontend app built in any language, ingesting messages into a stream processing framework that doesn't yet support Kafka, and scripting administrative actions.

It is a Confluent open-source project, included as part of their Confluent Platform.

Resources:

Introduction: https://github.com/confluentinc/kafka-rest/blob/master/README.md

Github Repository: https://github.com/confluentinc/kafka-rest

Complete documentation: https://docs.confluent.io/current/kafka-rest/docs/

116 questions
1
vote
2 answers

Confluent REST proxy API SSL handshake fails

I have a kafka cluster on docker using confluent images. I am using docker-compose to build the containers. When I tried to run the container it starts but can't communicate with any broker due to SSL handshake failed. I don't know if I miss some…
user3806586
  • 53
  • 9
1
vote
1 answer

Kafka rest api to get details of a consumer group

I am looking for a kafka rest api which will list the same details as kafka-consumer-groups.sh - - describe Would return, basically i am trying to get the details of offset of each partition and its lag for a particular consumer group
Abhishek
  • 519
  • 1
  • 6
  • 24
1
vote
1 answer

How to configure post webhook from BitBucket to Kafka REST Proxy?

I'm trying to post data(payload) from BitBucket to my Kafka instance in case of any BitBucket events (like Pushed, Forked, PR creating, Addition/Modification/Deletion of PR comments etc.) So, I've setup Kafka rest-proxy but I can't send data from…
Aryak Sengupta
  • 1,727
  • 2
  • 18
  • 23
1
vote
0 answers

Kafka Based Notification Engine for sending push notifications to Android and iOS apps

I am designing a Notification Engine (Java Based) which will be used as an one stop shop for various other systems to send notification to different channels (push notifications, SMS, email etc). I wish to build this system to use Kafka as a message…
bloomwood
  • 31
  • 2
  • 6
1
vote
2 answers

Confluent Kafka Rest Proxy - Avro Deserialization

I am trying to use Confluent Kafka REST Proxy to retrieve data in Avro format from one of my topics but unfortunately I get a deserialization error. I am querying the Kafka REST proxy using the following command curl -X GET -H "Accept:…
ypanag
  • 287
  • 5
  • 22
1
vote
1 answer

Confluent Rest Proxy Schem registry does not throw an error when sending missing property key

I register a Schema with the Schema registry as follows: { Prop1:val1, prop2:val2, prop3:val3} all fields above are optional When I send data payload via Rest Proxy that looks like the following: { "":val1, Prop2:val2, Prop3:val3 } I expect the…
Bharath Parlapalli
  • 75
  • 13
1
vote
1 answer

JSONParseException reading data using Kafka rest API

KAFKA TOPIC (test3) $ kafka-console-consumer --bootstrap-server broker:9092 --topic test3 --from-beginning "Can we write to a topic that does not exist?" "Can we write to a topic that does not…
vamsiampolu
  • 6,328
  • 19
  • 82
  • 183
1
vote
0 answers

Confluent-Kafka "Consumer instance not found" error even-if consumer instance is not timedout

I'm observing Consumer instance not found error, at the time of consumer registration even if consumer instance is not timedout. Using Confluent API's. Following are steps followed for this negative testing: Running a script for consumer…
Pooja
  • 481
  • 1
  • 8
  • 15
1
vote
1 answer

Kafka rest proxy: can I have two consumer instances in one consumer group

I'm using Kafka 0.10 REST proxy service, I have the following questions please: 1, Can I have two consumer instances in the same consume group? 2, How long the timeout of consumer instance once it was created or last used? 3, Is there Java or Python…
Jack
  • 5,540
  • 13
  • 65
  • 113
0
votes
0 answers

Kafka Rest Proxy not connecting to my Kafka Broker

I wanted to setup Kafka Rest proxy but ran into a problem. I have my kafka and zookeeper server running locally at localhost:9092 but when I am trying to setup my Kafka rest proxy using docker but the issue is when I am trying to start the container…
Umang Pachaury
  • 75
  • 8
0
votes
2 answers

Confluent REST Proxy POST to Topic responds with HTTP 415 Unsupported Media Type

I created a topic, test-topic, successfully. When I attempt to produce data for the topic with the curl command: curl -v -X POST -H "Content-Type: application/json" -H "Accept: application/json" \ --data…
Sam S.
  • 1
0
votes
0 answers

Kafka Rest Proxy v2 with docker image 7.3.1: error when producing message with schema validation

I am using docker image 7.31 to use Rest Proxy. I will use it to produce message to a confluent server Kafka. I am using V2 use REST PROXY to produce records, such as this below curl --location…
CHRISTOPHE LANNOY
  • 1
0
votes
0 answers

Kafka Rest Proxy: Topic metadata change

I have been trying to edit the retention.ms parameter for a kafka topic through a PUT method as follows: curl -X PUT -H "Content-Type: application/json" --data '{"configs":{"retention.ms":"10000"}}' http://localhost:8082/topics/t0/ However,…
Akshat Singh Rathore
  • 1
  • 3
0
votes
0 answers

Kafka Rest Proxy for automated testing: how to idiomatically clear out previous messages on topic?

We use the Kafka Rest Proxy for some of our automated testing making calls to it from Postman. A typical flow is is post event on input topic consume even on output topic in postman verify attributes of the output event The problem I am seeing is…
gsrunion
  • 379
  • 5
  • 14
0
votes
1 answer

Kafka REST Proxy API uneven distribution of messages

I have two partitions and messages in avro-format. I send the messages via the Kafka REST proxy API. I use the key for the messages. The key is a string. For example, there are my keys: 41:46-300, 41:45-300, 41:44-300, 41:43-300, 41:42-300. But the…
Demid Demidov
  • 11
  • 4
1 2 3
4
5 6 7 8