Stack Overflow
Stack Overflow

Questions tagged [kafka-rest]

Provides a RESTful interface to a Kafka cluster, so you can produce and consume messages, as well as perform administrative actions through any REST client.

Confluent REST Proxy

(From the official Readme.md linked below)

The Kafka REST Proxy provides a RESTful interface to a Kafka cluster. It makes it easy to produce and consume messages, view the state of the cluster, and perform administrative actions without using the native Kafka protocol or clients. Examples of use cases include reporting data to Kafka from any frontend app built in any language, ingesting messages into a stream processing framework that doesn't yet support Kafka, and scripting administrative actions.

It is a Confluent open-source project, included as part of their Confluent Platform.

Resources:

Introduction: https://github.com/confluentinc/kafka-rest/blob/master/README.md

Github Repository: https://github.com/confluentinc/kafka-rest

Complete documentation: https://docs.confluent.io/current/kafka-rest/docs/

116 questions
0
votes
1 answer

Produce Protobuf message using Rest proxy API

I am exploring options to integrate an ITSM system with Kafka. This needs Protobuf messages sent to Kafka via rest proxy API. I followed below two articles and set the request body and…
Viknesh
  • 5
  • 1
0
votes
0 answers

Less network calls for getting offsets of a Topic

This is the API call for getting topic offsets - GET /topics/(string:topic_name)/partitions/(int:partition_id)/offsets Here we have to give partition_id. Is there a way to get offsets for all partitions of a topic in one API call?
Aishwarya Sharma
  • 3
  • 2
0
votes
1 answer

Meaning of beginning_offset in Kafka topic GET /offsets API

This is a API from https://docs.confluent.io/5.5.11/kafka-rest/api.html#partitions Confluent documentation. GET /topics/(string:topic_name)/partitions/(int:partition_id)/offsets It returns this - HTTP/1.1 200 OK Content-Type:…
Aishwarya Sharma
  • 3
  • 2
0
votes
1 answer

How to consume data from Kafka topic from specific offset using REST Proxy for Kafka?

I am using Kafka Rest Proxy to produce as well as consume from Kafka topic using Azure APIM, hence the reason to use Rest Proxy. I can consume messages in a linear fashion using auto.offset.reset = earliest, so every call to Kafka Rest will give me…
Ravi P
  • 1
  • 2
0
votes
1 answer

Error when registering schema. Problem to produce AVRO messages via Kafka REST. Error_code: 40801

I'm following the confluent tutorial to produce messages (Produce and Consume Avro Messages), but when I post in the messages defining the schema registry, it gives the following error and I don't know how to continue, I've looked in several places…
souzatorquato
  • 53
  • 1
  • 6
0
votes
1 answer

Kafka Schema Registry - block messages that aren't accepted in the schema registry in kafka

I have a schema in kafka and I need that every time I post a post in this topic, the schema that I registered checks if it is in the same pattern that is being sent. My schema is: Curl post: curl -X POST -H "Content-Type:…
souzatorquato
  • 53
  • 1
  • 6
0
votes
2 answers

Kafka - increase partition count of existing topic through Confluent REST Api

I need to add partitions to an existing Kafka topic. I'm aware that it is possible to use the /bin/kafka-topics.sh script to achieve this, but I would prefer to do this through the Confluent REST api. As far as I see there is no documented endpoint…
code-gorilla
  • 2,231
  • 1
  • 6
  • 21
0
votes
1 answer

How can i consume avro data using kafka connect

I have created a connector for handling Avro data. I am able to publish data into a topic. But I am not getting the data into the output topic. I have checked the logs of connector and rest-proxy there is no error are showing. { "name":…
Renjith K R
  • 1
  • 1
0
votes
1 answer

how to change kafka rest proxy log level

i am using apache kafka rest proxy using docker compose and it makes heavy log file heavier than my massages size i there any parameter to set or something to do to disable this ? kafka-schema-registry: image:…
matin
  • 53
  • 1
  • 8
0
votes
0 answers

How to properly query Kafka REST Proxy?

I'm running a dockerized distribution of Confluent platform: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS …
Eugene Goldberg
  • 14,286
  • 20
  • 94
  • 167
0
votes
1 answer

Can't produce to Confluent Kafka topic through Kafka REST API

I have a trouble producing to Kafka topic through Rest API proxy. I have a running confluent kafka cluster in which I'd like to create a topic and produce a message to it using REST API. For that purpose I have followed the documentation and created…
Milenski
  • 5
  • 4
0
votes
1 answer

how can I make a kafka rest api call when the key is unquoted?

I'm trying to get kafka messages to an android app. I realized a little late in the game that the only way to do that is with Kafka Rest Proxy. I've set it all up but my key is unquoted leading to an error in deserialization. with…
xodust1
  • 15
  • 5
0
votes
0 answers

Correct flow for using Kafka REST Proxy

I am using Rest proxy in my application developed in Go and the steps for consuming topics are: Create a Consumer instance Subscribe to a topic get the records and delete the consumer instance Now the question I have is I want to have a consumer…
AR7
  • 366
  • 1
  • 16
0
votes
1 answer

Confluent Kafka REST Proxy Docker confluentinc/cp-kafka-rest:6.2.1 - couldn't start because of LOG4J errors

My config: kafka-rest-1: image: confluentinc/cp-kafka-rest:6.2.1 hostname: kafka-rest-1 container_name: kafka-rest-1 depends_on: - zookeeper-1 - broker-1 ports: - "8082:8082" environment: …
Alex Pristenskiy
  • 33
  • 2
  • 10
0
votes
1 answer

Using Kafka and Kafka REST Proxy server

I have two ends of an application: Python Flask backend, which communicates using Kafka in the normal way. A machine agent written in Go,installed on a client environment and communicates with Kafka only via Kafka REST proxy. Now the question is…
AR7
  • 366
  • 1
  • 16
1 2 3 4
5
6 7 8