Stack Overflow
Stack Overflow

Questions tagged [confluent-schema-registry]

Schema Registry provides a store, backed by Kafka, for storing and retrieving Avro schemas. It also provides Avro serializers for Kafka clients.

Schema Registry is part of Confluent Open Source Stream Processing Platform.

It provides a serving layer for your metadata. It provides a RESTful interface for storing and retrieving Avro schemas. It stores a versioned history of all schemas, provides multiple compatibility settings and allows evolution of schemas according to the configured compatibility setting. It provides serializers that plug into Kafka clients that handle schema storage and retrieval for Kafka messages that are sent in the Avro format.

1076 questions
5
votes
0 answers

Kafka Connect - JDBC Source Connector - Setting Avro Schema

How can I make Kafka Connect JDBC connector to predefined Avro schema ? It creates a new version when the connecter is created. I am reading from DB2 and putting into Kafka topic. I am setting schema name and version during creation but it does not…
usernotexist
  • 51
  • 1
  • 5
5
votes
3 answers

Unable to resolve confluent kafka dependencies in gradle?

While adding dependencies for confluent kafka in build gradle file, its unable to resolve it. compile group: 'io.confluent', name: 'kafka-avro-serializer', version: '4.0.0' compile group: 'io.confluent', name: 'kafka-schema-registry',…
ishant kulshreshtha
  • 71
  • 1
  • 6
5
votes
1 answer

Connector fails when schema registry's master changes

My source connector throws Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Error while forwarding register schema request to the master; error code: 50003 or Caused by:…
Holm
  • 2,987
  • 3
  • 27
  • 48
5
votes
1 answer

Error registering Avro schema: "string" RestClientException: Schema being registered is incompatible with an earlier schema;

I'm trying to send a message to my broker, using Avro schema, but "im always getting error: 2020-02-01 11:24:37.189 [nioEventLoopGroup-4-1] ERROR Application - Unhandled: POST -…
Rodrigo Batista
  • 148
  • 1
  • 10
5
votes
1 answer

How to replicate schema with Kafka mirror maker?

We are using the mirror maker to sync on-premise and AWS Kafka topics. How can a topic with its schema registered in on-premise be replicated exactly the same in other clusters (AWS in this case)? How Avro schema is replicated using mirror maker?
Tony
  • 671
  • 1
  • 9
  • 29
5
votes
1 answer

Unable to deserialize Kafka stream to pojo. Could not find class specified in writer's schema

Getting exception while reading from kafka topic: Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 1 Caused by: org.apache.kafka.common.errors.SerializationException: Could not find …
Yogesh Kumar Gupta
  • 163
  • 1
  • 3
  • 13
5
votes
2 answers

Unable to start Confluent Schema Registry

I have installed confluent platform in Ubuntu 16.04 machine and initially i have configured zookeeper, Kafka and ksql and started confluent platform. i am able to see the below message. root@DESKTOP-DIB3097:/opt/kafkafull/confluent-5.1.0/bin#…
sai krishna
  • 151
  • 2
  • 11
5
votes
1 answer

Schema-Registry rejects unchanged schema as incompatible

We have a kafka cluster running with Avro schemas stored in Confluent's Schema-registry. On a recent re-deploy of (one of) our streams applications we starting seeing incompatible schema errors on a single topic (EmailSent). This is the only…
Freestyle076
  • 1,548
  • 19
  • 36
5
votes
1 answer

One or more schemas per topic when using Schema Registry with Kafka, and Avro...?

There is somethigng I'm trying to understand about how Avro-serialized messages are treated by Kafka and Schema Registry - from this post I've understood the schema ID is stored in an predictable place in each message so it seems that we can have…
Michal Pawluk
  • 561
  • 1
  • 5
  • 11
5
votes
1 answer

Avro fails to deserialize message with updated schema

I have a schema which has been updated to include a new field. I'm using avro reflection and the confluent schema registry to deserialize/serialize data like so: Serialization: Schema schema = REFLECT_DATA.getSchema(value.getClass()); try { int…
cscan
  • 3,684
  • 9
  • 45
  • 83
5
votes
1 answer

Kafka with Confluent Kubernetes Helm Charts = Schema Registry WakeupException

My Main Question: Why is the schema-registry crashing? Peripheral Question: Why are two pods launching for each of zookeeper/kafka/schema-registry if I've configured one server for each? Does everything thing else look basically right? ➜ helm repo…
clay
  • 18,138
  • 28
  • 107
  • 192
5
votes
1 answer

Metadata information from kafka

I am new to Confluent/Kafka and I want to find metadata information from kafka I want to know list of producers list of topics schema information for topic Confluent version is 5.0 What are classes (methods) that can give this information? Are…
user1393608
  • 1,299
  • 3
  • 16
  • 29
5
votes
1 answer

Spring Embedded Kafka + Mock Schema Registry: State Store ChangeLog Schema not registered

I'm building an integration test for our kafka system using the Spring Embedded Kafka Broker, with a MockSchemaRegistryClient. I am building a test for one of our Stream topologies, built using the Streams API (KStreamBuilder). This particular…
Freestyle076
  • 1,548
  • 19
  • 36
5
votes
4 answers

make avro schema from a dataframe - spark - scala

i want to write a stream of dataframes into kafka in avro format; i think that i should publish the dataframe schema to schema registry as an avro schema then write the stream of DF to kafka specifying that schema as an option. So i need to know…
sghayth
  • 73
  • 1
  • 8
5
votes
1 answer

What is the reasoning behind Kafka Connect Schemas?

We are writing a custom sink connector for writing content of a topic with avro messages to a CEPH storage. To do this we are provided with SinkRecords which have a Kafka Connect schema which is a mapped version of our avro schema. Since we want to…
Chris W.
  • 2,266
  • 20
  • 40
1 2 3
…
71 72