Stack Overflow
Stack Overflow

Questions tagged [kafka-rest]

Provides a RESTful interface to a Kafka cluster, so you can produce and consume messages, as well as perform administrative actions through any REST client.

Confluent REST Proxy

(From the official Readme.md linked below)

The Kafka REST Proxy provides a RESTful interface to a Kafka cluster. It makes it easy to produce and consume messages, view the state of the cluster, and perform administrative actions without using the native Kafka protocol or clients. Examples of use cases include reporting data to Kafka from any frontend app built in any language, ingesting messages into a stream processing framework that doesn't yet support Kafka, and scripting administrative actions.

It is a Confluent open-source project, included as part of their Confluent Platform.

Resources:

Introduction: https://github.com/confluentinc/kafka-rest/blob/master/README.md

Github Repository: https://github.com/confluentinc/kafka-rest

Complete documentation: https://docs.confluent.io/current/kafka-rest/docs/

116 questions
2
votes
1 answer

How to convert a curl command to Resttemplate

i want to convert a curl command which post a message into Kafka Rest Proxy. The curl command is curl -u username:password --request POST --url https://kafka-rest-proxy-*****.com/topics/test-topic --header 'accept: application/vnd.kafka.v2+json,…
Bunny
  • 111
  • 2
  • 11
2
votes
2 answers

docker schema registry and kafka rest cannot up

both zookeeper and Kafka communicated well and up running. not sure why the schema registry and Kafka rest cannot up. below is the docker-compose file. this is the schema and rest docker-compose file. error on this docker [main] ERROR…
user2201789
  • 1,083
  • 2
  • 20
  • 45
2
votes
1 answer

CURL request to kafka-rest

I have executed HTTP POST on kafka-rest using curl call. one request is successful but another request (with different json) is returned 422 error ({"error_code":422,"message":"Unrecognized field: receiver"}). Working request curl -X POST -H…
Abdul Rahuman Seeni Mohamed
  • 33
  • 1
  • 2
  • 7
2
votes
1 answer

How to create topics from command line in Kafka REST proxy

I am using Kafka rest proxy, but not whole Confluent Platform just Kafka rest with my Kafka brokers. But I am not able to create topics from command line by the following command. bin/kafka-topics.sh --create --bootstrap-server localhost:9092…
Wajahat Lateef
  • 77
  • 1
  • 8
2
votes
1 answer

Problems running Kafka with Docker Compose in Windows

I'm trying to run Kafka locally on Windows 10 Pro and Docker Desktop (not toolbox). Everything seems to work perfectly, but I can't reach Kafka with my application and neither use kafka rest (http://localhost:8082/topics |…
Javarian
  • 127
  • 2
  • 13
2
votes
3 answers

Failed to deserialize data for topic

I'm using confluent cp-all-in-one project configuration from here: https://github.com/confluentinc/cp-docker-images/blob/5.2.2-post/examples/cp-all-in-one/docker-compose.yml I'm POST-ing a message to http://localhost:8082/topics/zuum-positions with…
Cristian Boariu
  • 9,603
  • 14
  • 91
  • 162
2
votes
2 answers

Why base64 encode/decode in Kafka REST Proxy?

Producer serializes the message and send them to Broker in byte arrays. And Consumers deserializes those byte arrays. Broker always stores and passes byte arrays. This is how I understood. But when you use REST Proxy in Kafka, Producer encodes the…
Jin Lee
  • 3,194
  • 12
  • 46
  • 86
2
votes
2 answers

Could not find artifact io.confluent:kafka-rest-parent:pom:5.4.0-SNAPSHOT and 'parent.relativePath' points at no local

I am using maven 3 to run the application but I am getting the following error: [ERROR] [ERROR] Some problems were encountered while processing the POMs: [FATAL] Non-resolvable parent POM for io.confluent:kafka-rest:[unknown-version]: Could not find…
ARAVINTHAN
  • 21
  • 2
  • 4
2
votes
1 answer

Exception in Kafka REST Proxy "Schema being registered is incompatible with an earlier schema" when trying to post record with nullable key schema

I'm trying to post this message to Kafka Rest: { "key_schema": "[\"null\",\"long\"]", "value_schema":…
provisota
  • 1,390
  • 1
  • 14
  • 14
2
votes
2 answers

Processing huge records gives OutOfMemoryException - Kafka REST proxy

I'm calling Kafka using the confluent REST API proxy. I'm reading a CSV file, creating an object out of all the records there (about 4 million records) and sending a request to the REST proxy. I keep on getting an OutOfMemory exception. The exact…
Gaurav Goenka
  • 152
  • 12
2
votes
3 answers

How to limit number of records in Kafka-consumer

I am using confluent Kafka-rest product to consume records from a topic. My intention is to consume only first 100 records from topic. I am using the following REST API to fetch records GET /consumers/testgroup/instances/my_consumer/records How to…
Achaius
  • 5,904
  • 21
  • 65
  • 122
2
votes
0 answers

CORS issue with Confluent kafka rest proxy

I am facing some problem in my own Kafka(confluent) setup. Please find the below issue CORS Issue :- Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. I…
Priyabrata
  • 629
  • 1
  • 8
  • 23
2
votes
2 answers

How to indicate consumer offset in Kafka rest API

I'm using Kafka 0.10 REST API. I just used an HTTP object in Java to invoke the Kafka REST API (such as the curl commands). I need to indicate the consumer offset when I consume the message, otherwise it reads from the beginning or the latest, but I…
Jack
  • 5,540
  • 13
  • 65
  • 113
1
vote
1 answer

What schema format supports Kafka REST v3?

I revealed that tutorial schema version ref from tutorial doesn't work in kafka version3. In additional the same result I received if I try to send the same tutorial via kafka-rest v2 from cp-kafka-rest:6.2.1 instead of cp-kafka-rest:6.1.0 An…
Irina
  • 939
  • 1
  • 8
  • 26
1
vote
1 answer

mTLS on Kafka rest proxy

I'm trying to apply mTLS security on Kafka rest proxy and no luck. The model that I'm looking for is as below. Browser --https://host:443/--> Kafka Rest proxy --kerberos--> Kafka Brokers Rest-proxy to Kafka brokers is working fine, but client to…
Raman
  • 35
  • 1
  • 7
1
2
3 4 5 6 7 8