Stack Overflow
Stack Overflow

Questions tagged [confluent-schema-registry]

Schema Registry provides a store, backed by Kafka, for storing and retrieving Avro schemas. It also provides Avro serializers for Kafka clients.

Schema Registry is part of Confluent Open Source Stream Processing Platform.

It provides a serving layer for your metadata. It provides a RESTful interface for storing and retrieving Avro schemas. It stores a versioned history of all schemas, provides multiple compatibility settings and allows evolution of schemas according to the configured compatibility setting. It provides serializers that plug into Kafka clients that handle schema storage and retrieval for Kafka messages that are sent in the Avro format.

1076 questions
2
votes
0 answers

How can I encrypt the userInfoConfig parameter while using the kafka-schema-registry-maven-plugin provided by io.confluent?

My spring boot maven application connects to the schema registry in order to download the schema files as below pom.xml io.confluent kafka-schema-registry-maven-plugin …
creationk
  • 39
  • 9
2
votes
1 answer

Which is better auto register schema or manual register in confluent schema registry?

I'm trying to understand the confluent schema registry and evolution of schema in it. I couldn't find which is better among the following. What is preferred to use and when ? What are the pros and cons of the two approaches ? Producer Auto…
avocado
  • 39
  • 1
  • 3
2
votes
1 answer

Difference in key/value subjects in Schema Registry

I'm a newbie in schema registry in Kafka, but I cannot realize what's the difference between kafka-value and kafka-key in schema registry? I registered a new topic: test Now, if I make a request to /subjects I have: test test-value So my…
Oleg L
  • 161
  • 1
  • 14
2
votes
2 answers

Avro fingerprint schema

I am using Confluent kafka in C# to consume Messages, these messages are formatted as hexadecimal strings from which the Schema fingerprint is extracted. How to get the schema from the schema fingerprint in C#? am I missing something?
Elias Ghali
  • 823
  • 1
  • 13
  • 29
2
votes
1 answer

Configuring Spark Structured Streaming with authenticated Confluent Schema Registry

I’m using a Kafka Source in Spark Streaming to receive records generated using Datagen in Confluent Cloud. I intend to use Confluent Schema Registry, Currently, this is the exception I am facing : * Exception in thread…
Aditi Sinha
  • 29
  • 4
2
votes
1 answer

Spring Kafka ConsumerConfig incorrectly lists StringDeserializer instead of KafkaAvroDeserializer for value.deserializer

When starting Spring Boot 2.4.5 with spring-kafa, value.deserializer value is shown as StringDeserializer instead of KafkaAvroDeserializer: 2021-05-12 13:46:05.313 INFO 12632 --- [ main] o.a.k.clients.consumer.ConsumerConfig :…
kmansoor
  • 4,265
  • 9
  • 52
  • 95
2
votes
1 answer

unable to retrieve kafka confluent schema regsitry registry via `@kafkajs/confluent-schema-registry`

I'm using @kafkajs/confluent-schema-registry in a typescript node app, attempting to pull down schemas. I have done this in Java with the analogus confluent schema registry java library w\o issue. When creating the schema registry client, I don't…
Stealth Rabbi
  • 10,156
  • 22
  • 100
  • 176
2
votes
1 answer

Spring cloud contract tests for avro messages using Schema Registry

I am checking out the spring docs and spring github, and I can see some very basic examples of async kafka messaging (producer and consumer) that communicate via json messages and Kafka topics, but it seems that there are no examples for more…
Vassilis
  • 914
  • 8
  • 23
2
votes
0 answers

Validating schema locally with schemas from confluent schema registry

Previously, I used SchemaLoader to load json schema files I had locally and validated JSON objects against them. Some of my schemas have refs to other local files. I'm now migrating to the confluent schema registry and I've registered my schemas in…
Stealth Rabbi
  • 10,156
  • 22
  • 100
  • 176
2
votes
2 answers

Issue for Integrating Hudi with Kafka using Avro Schema

I am trying to integrate Hudi with Kafka topic. Steps followed : Created Kafka topic in Confluent with schema defined in schema registry. Using kafka-avro-console-producer, I am trying to produce data. Running Hudi Delta Streamer in continuous mode…
Prashant
  • 21
  • 5
2
votes
2 answers

Having issues with using schema-registry:download

I am using the schema registry download maven from here https://docs.confluent.io/platform/current/schema-registry/develop/maven-plugin.html, but I am not able to download the schema into my local project. Here is my plugin content. …
Shan
  • 177
  • 3
  • 12
2
votes
0 answers

Using Json format in Confluent Platform Schema Registry?

I'm trying to put several event types in the same Kafka topic using the JSON format, but in the Producer implementation I'm always getting org.apache.kafka.common.errors.SerializationException: Error serializing JSON message. Seems that the…
ddss
  • 393
  • 1
  • 2
  • 7
2
votes
1 answer

Is it Possible to send a Java Object to C# App using Kafka

Is it possible to send send a Java Object (lets say user) to a topic that is consumed and serialised to a User Object in C#? Lets say I have the following avro schema built from a a Java PoJo (Fields are Name and Age) { "namespace":…
MetaCoder
  • 368
  • 5
  • 22
2
votes
0 answers

Serialisation error for Kafka avro consumer using Spring boot

I have created a Kafka Avro producer and consumer using spring boot as two different projects. While consuming the data I am getting the following exception. Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing…
SHILPA
  • 57
  • 1
  • 3
  • 11
2
votes
1 answer

Should Avro be used to on both the key and value in Kafka?

We're working to setup a Kafka cluster and exploring the use of Avro but I haven't been able to find guidance on if Avro should be used on both the key and value of a Kafka message. I've explored both use cases and I'm not really seeing the benefit…
electric_oasis
  • 23
  • 3
1 2 3
…
71 72