Stack Overflow
Stack Overflow

Questions tagged [confluent-schema-registry]

Schema Registry provides a store, backed by Kafka, for storing and retrieving Avro schemas. It also provides Avro serializers for Kafka clients.

Schema Registry is part of Confluent Open Source Stream Processing Platform.

It provides a serving layer for your metadata. It provides a RESTful interface for storing and retrieving Avro schemas. It stores a versioned history of all schemas, provides multiple compatibility settings and allows evolution of schemas according to the configured compatibility setting. It provides serializers that plug into Kafka clients that handle schema storage and retrieval for Kafka messages that are sent in the Avro format.

1076 questions
0
votes
0 answers

Kafka Connect Avro Schema Registry - Missing logicalType

I've been writing a custom kafka connector plugin. For this, I've to create a schema dynamically from the metadata file and then read each record in data file and publish individual record to a topic. Below is sample code to create schema: Create…
user123
  • 281
  • 1
  • 3
  • 16
0
votes
1 answer

Kafka console consumer not consuming from topic

We have a server that is handling the production and consumption of messages. We have 4 laptops, all macs with confluent all running the same command line... ./kafka-avro-console-consumer --from-beginning --bootstrap-server…
Peej
  • 3
  • 1
0
votes
1 answer

Schema Registry persistence after reboot

I just finished this tutorial to use Kafka and Schema Registry :http://cloudurable.com/blog/kafka-avro-schema-registry/index.html I also played with Conlfuent Platform : https://docs.confluent.io/current/installation/installing_cp.html Everything…
user954156
  • 468
  • 2
  • 6
  • 17
0
votes
1 answer

Should use Kafka Schema Registry server

When I used Schema registry server, I found I could not change schema in the same topic anymore. but without schema registry each record has it's own schema inside, so it's easy to change schema anytime. So how can I change the schema when I'm…
Jack
  • 5,540
  • 13
  • 65
  • 113
0
votes
1 answer

Kafka - consuming from spark

I followed this document, and It works well. Now I tried to consuming connector data from spark. Is there any reference I can use? Since I use confluent, It's much different from original kafka reference document. It's some code that I've used so…
J.Done
  • 2,783
  • 9
  • 31
  • 58
0
votes
1 answer

Avro schema update with two schema in one avro file

I have one avro file with first schema then I updated the schema that appends to the same file. So now I have two schemas in one file. How does avro handle this scenario. Will I have any new fields add in the file or will I loose any data while…
buckeyeosu
  • 45
  • 8
0
votes
0 answers

Issues on running Confluent Schema Registry

I have created a kafka cluster in separate three centOS machine. All the Zookeepers and kafka server is running properly. When i try to run schema registry then the host zookeeper got down, therefore schema registry raised a error like error…
user8875847
  • 1
0
votes
1 answer

does using schema registry make a significant difference?

if we are using schema registry in Kafka, is it required for every producer to send current version of Kafka every time it sends the record to broker? if yes, what is the meaning of this extra overhead because we were already sending schema in every…
amrit sharma
  • 197
  • 1
  • 2
  • 11
0
votes
3 answers

Kafka connector exception ERROR Failed to send HTTP request to endpoint: http://localhost:8081/subjects/jdbc-source-accounts-value/versions

I'm using Confluent Platform 3.3 as a Kafka connector , while starting the connector using the below command, ./bin/connect-standalone ./etc/schema-registry/connect-avro-standalone.properties…
Raju
  • 448
  • 1
  • 9
  • 24
0
votes
2 answers

Can I use confluent Schema Registry to generate schema less avro msgs from flat file?

I was wondering can I use Confluent Schema registry to generate (and then send it to kafka) schema less avro records? If yes can somebody please share some resources for it? I am not able to find any example on Confluent website and Google. I have…
Explorer
  • 1,491
  • 4
  • 26
  • 67
0
votes
3 answers

Schema Registry container: Server died unexpectedly when launching using docker-compose

I have written docker-compose.yml file to create the following containers: Confluent-Zookeeper Confluent-Kafka Confluent-Schema Registry I want a single docker-compose file to spun up the necessary containers, expose required ports and…
Arman Koradia
  • 11
  • 1
  • 3
0
votes
2 answers

How to run schema-registry in master slave mode

As of now I am running schema-registry as a single node. To provide fault tolerance, I want to run schema-registry in master slave mode. One node for master and one node for slave. Do we need to add any extra property to identify a node as…
Renukaradhya
  • 812
  • 2
  • 19
  • 31
0
votes
1 answer

Registering AVRO schema with confluent schema registery

Can AVRO schemas be registered with confluent schema registry service ? As per readme on github https://github.com/confluentinc/schema-registry Every example uses a JSON schema with a single field and type without any name. I am trying to store…
tarunkumar
  • 861
  • 2
  • 15
  • 30
0
votes
1 answer

Confluent schema-registry 2.0.1 version with SSL configuration

I need to use confluent schema-registry to connect to 0.9 kafka. But for schema-registry 2.0.1 version I didn't see SSL configuration available. Is there a way to enable ssl for schema-registry and kafka-rest to talk to 0.9 kafka?
edi
  • 223
  • 5
  • 12
-1
votes
2 answers

Extract all schemaIDs of all messages in a kafka topic without the need to consume all messages

I'm wondering if there's way (kafka API/tool) to return list of schemaIds used by messages under a topic from within kafka and/or schema registry. I have a quick soluiton to consume all messages to extract from outside of kafka. However, it's kind…
Jin Ma
  • 169
  • 2
  • 12
1 2 3
…
71
72