Stack Overflow
Stack Overflow

Questions tagged [confluent-schema-registry]

Schema Registry provides a store, backed by Kafka, for storing and retrieving Avro schemas. It also provides Avro serializers for Kafka clients.

Schema Registry is part of Confluent Open Source Stream Processing Platform.

It provides a serving layer for your metadata. It provides a RESTful interface for storing and retrieving Avro schemas. It stores a versioned history of all schemas, provides multiple compatibility settings and allows evolution of schemas according to the configured compatibility setting. It provides serializers that plug into Kafka clients that handle schema storage and retrieval for Kafka messages that are sent in the Avro format.

1076 questions
4
votes
0 answers

how to use AWS Eventbridge Schema Registry code binding typescript Marshaller?

I am new to eventbridge schema registry and typescript. After I set a registry and schema in aws console, and download the code bindings. I am wondering how to use this Marshaller class? I know checking schema basically is to encode and decode data,…
Will Zefeng Qiu
  • 41
  • 1
4
votes
1 answer

Kafka Confluent InvalidConfigurationException: Unauthorized; error code: 401

I am reading a schema using SpecificAvroSerde from the kafka Confluent schema registry. but I am getting this error below: org.apache.kafka.common.errors.InvalidConfigurationException: Unauthorized; error code: 401 [TestStreamProcess-StreamThread-1]…
Charmee Lee
  • 123
  • 1
  • 9
4
votes
1 answer

Ensuring consistency with Kafka Schema and OpenAPI specification

A number of API/microservices provide access to critical resources including Kafka topics. The API/microservice messages are validated using an OpenAPI specification that defines the API/microservice contract. Once the microservice validates the…
Eric Broda
  • 6,701
  • 6
  • 48
  • 72
4
votes
1 answer

Kafka Rest Proxy JSON schema validation

I want to produce to a kafka topic via Kafka Rest Proxy. I've created a JSON schema in the schema-registry and I would like that all messages are validated against the registered schema and rejected if they don't match the Schema. My Schema { …
lloiacono
  • 4,714
  • 2
  • 30
  • 46
4
votes
1 answer

How to retrieve AVRO Schema from Schema Registry from a message in a Topic from java consumer

We are using Confluent Kafka with schema registry. We have more than 40 topics. Our application writes avro messages to these topics, using the schema from the registry. From what I understand, when using the registry, the message doesn't…
adbdkb
  • 1,897
  • 6
  • 37
  • 66
4
votes
0 answers

Confluent.Kafka SchemaRegistry: reusing Avro schemas?

Avro supports schema reusing. One might put one custom type in file a.avsc and reuse it in the another file b.avsc. That's fatefully important for more or less complicated projects to avoid having identical entities under different namespaces and…
Zazaeil
  • 3,900
  • 2
  • 14
  • 31
4
votes
1 answer

How to solve issue in "schema.registry.url" while connecting elastic search with Kafka Connect?

I am trying to connect Kafka Connect with elastic search sink. I am not using in confluent but in standalone mode. This is my elasticsearch connector…
Afzal Abdul Azeez
  • 195
  • 1
  • 13
4
votes
1 answer

Is it possible to deserialize Avro message(consuming message from Kafka) without giving Reader schema in ConfluentRegistryAvroDeserializationSchema

I am using Kafka Connector in Apache Flink for access to streams served by Confluent Kafka. Apart from schema registry url ConfluentRegistryAvroDeserializationSchema.forGeneric(...) expecting 'reader' schema. Instead of providing read schema I want…
Sparkle8
  • 225
  • 3
  • 13
4
votes
1 answer

What are strategies for sharing Avro schema across multiple topics or other schema?

Apologies if this is already covered in Kafka's documentation or a guide, and I'd be thankful if somebody could point me to that. I've found a lot of documents and articles that cover the basics of how to use Avro with Kafka and the Schema Registry,…
Bill Bushey
  • 143
  • 1
  • 6
4
votes
2 answers

how to share avro schema definitions across teams

Kafka schema-registry provides a nice way to serialize and deserialize the data from Kafka using common data contract. However, the data contract (.avsc file) is the glue between the producer and consumer(s). Once producer makes the .avsc file, it…
Sahas
  • 3,046
  • 6
  • 32
  • 53
4
votes
1 answer

How to remove schema from confluent schema registry with specific id?

I'm using the community version of confluent-schema-registry. As far as I understand, too many schema under same topics may cause compatibility issues later. I tried hard to delete useless schemas in the schema registry with the methods mentioned in…
Sophie
  • 119
  • 2
  • 6
4
votes
2 answers

Kafka connector and Schema Registry - Error Retrieving Avro Schema - Subject not found

I have a topic that will eventually have lots of different schemas on it. For now it just has the one. I've created a connect job via REST like this: { "name":"com.mycompany.sinks.GcsSinkConnector-auth2", "config": { "connector.class":…
Dylan
  • 1,306
  • 1
  • 11
  • 29
4
votes
2 answers

Why kafka-avro-console-producer doesn't honour the default value for the field?

Although default is defined for a field, kafka-avro-console-producer ignores it completely: $ kafka-avro-console-producer --broker-list localhost:9092 --topic test-avro \ --property schema.registry.url=http://localhost:8081 --property…
hdjur_jcv
  • 686
  • 1
  • 12
  • 30
4
votes
2 answers

Add schema to Schema Registry with a specific Id

We have been using Confluent Schema Registry with KafkaStreams for over a year now and everything has been working well; until yesterday. In an UAT environment we seem to have had a Schema subject deleted and one of our applications started failing…
DVS
  • 783
  • 7
  • 25
4
votes
1 answer

Avro union: forward compatibility?

Our system consists of multiple microservices that emit and consume events encoded in avro format (see schema at the bottom). A particular use case is the following: Service A emits an event (of type InvoiceEvents) on topic T1 and Services B and C…
Vassilis
  • 914
  • 8
  • 23
1 2 3
…
71 72