Stack Overflow
Stack Overflow

Questions tagged [confluent-schema-registry]

Schema Registry provides a store, backed by Kafka, for storing and retrieving Avro schemas. It also provides Avro serializers for Kafka clients.

Schema Registry is part of Confluent Open Source Stream Processing Platform.

It provides a serving layer for your metadata. It provides a RESTful interface for storing and retrieving Avro schemas. It stores a versioned history of all schemas, provides multiple compatibility settings and allows evolution of schemas according to the configured compatibility setting. It provides serializers that plug into Kafka clients that handle schema storage and retrieval for Kafka messages that are sent in the Avro format.

1076 questions
9
votes
2 answers

How to use Spring-Kafka to read AVRO message with Confluent Schema registry?

How to use Spring-Kafka to read AVRO message with Confluent Schema registry? Is there any sample? I can't find it in official reference document.
Casel Chen
  • 497
  • 2
  • 8
  • 19
9
votes
3 answers

Kafka Stream with Avro in JAVA , schema.registry.url" which has no default value

I have the following configuration for my Kafka Stream application Properties config = new Properties(); config.put(StreamsConfig.APPLICATION_ID_CONFIG,this.applicaionId); …
Benny Chan
  • 333
  • 1
  • 4
  • 10
8
votes
2 answers

Kafka : ClassCastException: class org.apache.avro.generic.GenericData$Record cannot be cast to class

I have this exception in the consumer when trying to cast the record.value() into java object : ClassCastException: class org.apache.avro.generic.GenericData$Record cannot be cast to class [...].PublicActivityRecord…
Rodjf
  • 177
  • 1
  • 12
8
votes
1 answer

kafka schema.registry.url was supplied but isn't a known config

Trying to publish the json message on topic using schema registry but getting below error. followingspring boot approach .. The configuration 'schema.registry.url' was supplied but isn't a known config application yml fle server: port:…
rakesh kandukuri
  • 195
  • 1
  • 11
8
votes
2 answers

Kafka Sink Connector fails: Schema not found; error code: 40403

I have a sink connector with the following configuration { "name": "sink-test-mariadb-MY_TOPIC", "config": { "connector.class":"io.confluent.connect.jdbc.JdbcSinkConnector", "tasks.max":"10", …
Giorgos Myrianthous
  • 36,235
  • 20
  • 134
  • 156
8
votes
2 answers

what is key schema in schema registry?

I don't have exact idea of key schema, that what it is, and why it must be used as key is auto-generated and we just pass a value(message). For value, we pass a schema to the AVRO Serialiser and the serialiser gets it's schema id from schema…
chitwan
  • 263
  • 2
  • 4
  • 15
8
votes
3 answers

Kafka Confluent error - java.net.BindException: Address already in use

I am running Kafka via Confluent platform. I have followed the steps as per mentioned, java.net.BindException: Address already in use As per documentation here, https://docs.confluent.io/2.0.0/quickstart.html#quickstart start zookeeper, $…
Stella
  • 1,728
  • 5
  • 41
  • 95
8
votes
1 answer

Is there any way to use confluent schema registry with kafka-node module?

I have implemented Avro schema in node.js with schema being sent with the message payload. And it is working fine. I am looking if there is any way I can use schema-registry with Kafka-node module. I have explored but was not successful in finding…
Nandish Kotadia
  • 441
  • 1
  • 6
  • 21
7
votes
2 answers

how to enforce schema validation in kafka

I'm using Kafka & Schema-registry. Defined a schema, using confluent's KafkaAvroSerializer on the producer side. Everything works fine. On the other hand, if a producer publishes the event without adhering to the schema, it gets published without…
Sahas
  • 3,046
  • 6
  • 32
  • 53
7
votes
1 answer

Getting schema deserialisation error : Invalid default for field

While running kafka consumer, i am getting following error: Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 13 Caused by: org.apache.avro.AvroTypeException: Invalid default for field key_id:…
SunilS
  • 2,030
  • 5
  • 34
  • 62
7
votes
2 answers

Use Avro in KafkaConnect without Confluent Schema Registry

We have vanilla apache Kafka setup in current infrastructure and we started logging some data that we want to process using Kafka Connect. Currently we use Avro for our message format, but there's no Schema Registry in our infrastructure. In future,…
Martin Macak
  • 3,507
  • 2
  • 30
  • 54
7
votes
1 answer

AVRO schema's JSON looks valid but returns invalid from Kafka Schema Registry

I am trying to register an AVRO schema to Schema Registry. The schema contains a record and some fields. I post the schema as JSON to Schema Registry REST API and although the JSON look fine the server returns curl :…
null
  • 429
  • 1
  • 3
  • 10
7
votes
1 answer

How to ensure constant Avro schema generation and avoid the 'Too many schema objects created for x' exception?

I am experiencing a reproducible error while producing Avro messages with reactive kafka and avro4s. Once the identityMapCapacity of the client (CachedSchemaRegistryClient) is reached, serialization fails with java.lang.IllegalStateException: Too…
kostja
  • 60,521
  • 48
  • 179
  • 224
7
votes
3 answers

Can a C# model be serialized as an AVRO JSON Schema?

I have found some code here https://learn.microsoft.com/en-us/azure/hdinsight/hdinsight-dotnet-avro-serialization#Scenario2 that does the reverse of what I need: //Define the schema in JSON const string Schema = @"{ ""type"":""record"", …
joelnet
  • 13,621
  • 5
  • 35
  • 49
6
votes
1 answer

How can I register a protobuf schema with references in other packages in Kafka schema registry?

I'm running Kafka schema registry version 5.5.2, and trying to register a schema that contains a reference to another schema. I managed to do this when the referenced schema was in the same package with the referencing schema, with this curl…
omer
  • 1,242
  • 4
  • 18
  • 45
1
2
3
…
71 72