Stack Overflow
Stack Overflow

Questions tagged [confluent-schema-registry]

Schema Registry provides a store, backed by Kafka, for storing and retrieving Avro schemas. It also provides Avro serializers for Kafka clients.

Schema Registry is part of Confluent Open Source Stream Processing Platform.

It provides a serving layer for your metadata. It provides a RESTful interface for storing and retrieving Avro schemas. It stores a versioned history of all schemas, provides multiple compatibility settings and allows evolution of schemas according to the configured compatibility setting. It provides serializers that plug into Kafka clients that handle schema storage and retrieval for Kafka messages that are sent in the Avro format.

1076 questions
3
votes
3 answers

How to use camel-avro-consumer & producer?

I dont see an example of how to use camel-avro component to produce and consume kafka avro messages? Currently my camel route is this. what should it be changed in order to work with schema-registry and other props like this using camel-kafka-avro…
driven_spider
  • 495
  • 4
  • 16
3
votes
1 answer

Confluent Schema Registry Docker image not exposing port 8081 outside the container

I'm running the following container using the docker image for the Confluent Schema Registry. Everything runs fine inside the container meaning I can run a shell command inside the container against localhost:8081/subjects and get an empty list back…
emirhosseini
  • 545
  • 2
  • 8
  • 17
3
votes
4 answers

Spring Kafka, Spring Cloud Stream, and Avro compatibility Unknown magic byte

I have a problem deserializing messages from Kafka topics. The messages have been serialized using spring-cloud-stream and Apache Avro. I am reading them using Spring Kafka and trying to deserialise them. If I use spring-cloud to both produce and…
robjwilkins
  • 5,462
  • 5
  • 43
  • 59
3
votes
1 answer

Kafka Avro Console Consumer not working after enabling SSL encryption and authentication

I have recently enabled 2-way authentication on my Kafka Cluster. I am trying to consume messages from a topic in Avro format using kafka-avro-console-consumer --bootstrap-server kafka-host:9092 --topic my_topic --consumer.config…
Giorgos Myrianthous
  • 36,235
  • 20
  • 134
  • 156
3
votes
1 answer

How to update KSQL stream definition dynamically based on schema-registry

I created a KSQL stream based on schema-registry by following this post. The Kafka JDBC connector updates a latest schema in schema-registry. The new stream gets created with the latest schema, but existing stream sill in the oldest schema. I don't…
Achaius
  • 5,904
  • 21
  • 65
  • 122
3
votes
2 answers

Confluent: ERROR Failed to run query for table TimestampIncrementingTableQuerier mysql-jdbc

I am trying to use mode timestamp with MySQL, with limited rows as my table size is 2.6 GB. Here are the connector properties that I am using: { "name": "jdbc_source_mysql_registration_query", "config": { …
kotesh
  • 41
  • 1
  • 4
3
votes
1 answer

How to run two instances of schema registry

I am trying to run Kafka in cluster mode using two instances of schema registry but I am not quite sure how to configure the second instance so that it takes over in case the first one is down. Here's the properties file for the first…
Giorgos Myrianthous
  • 36,235
  • 20
  • 134
  • 156
3
votes
2 answers

Kafka - Schema Registry unavailable

I downloaded Confluent Platform (on local laptop MacOS) and followed instructions for starting kafka: https://docs.confluent.io/current/schema-registry/docs/intro.html ~/kafka/confluent-4.0.0/bin » confluent start schema-registry …
Joe
  • 11,983
  • 31
  • 109
  • 183
3
votes
1 answer

Where does Confluent's kafka-avro-console-consumer write its log files?

I am running Confluent's kafka-avro-console-consumer as described in the quickstart tutorial: kafka-avro-console-consumer --topic test --zookeeper localhost:2181 --from-beginning However, I have created a separate user kafka to run this command. I…
Elias Strehle
  • 1,722
  • 1
  • 21
  • 34
3
votes
1 answer

Kafka Streams JDBC Source Long Incompatibility

Problem: After setting up a Kafka pipeline that pulls data in using a Kafka Connect JDBC source with Avro serializers and deserializers, once I try to read that data into a KStream using a Kafka Streams Java app, I get the following…
Allen Underwood
  • 517
  • 5
  • 11
3
votes
1 answer

Kafka Streams - Unknown magic byte on GenericAvroSerde

when trying to stream Avro data with Kafka Streams, I came across this error: Exception in thread "StreamThread-1" org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1 Caused by:…
Tim.G.
  • 299
  • 4
  • 16
3
votes
2 answers

Confluent schema registry fails on start with NoSuchMethodError

Exception in thread "main" java.lang.NoSuchMethodError: io.confluent.rest.Application.parseListeners(Ljava/util/List;ILjava/util/List;Ljava/lang/String;)Ljava/util/List; at…
Nayan Sharma
  • 1,823
  • 18
  • 19
2
votes
1 answer

What is the cause of unexpected bytes in java KafkaProtobufSerializer value?

I have two applications, a node app, using @kafkajs/confluent-schema-registry library and a java app using standard KafkaProtobufSerializer. Topic is schema bound with protobuf. When I analyze the contents of the topic when the two apps serialize…
user177933
  • 63
  • 5
2
votes
1 answer

Adding packages to memgraph transformation

I am writing a memgraph transformation in python. When I import modules such as "requests" or "networkx", the transformation works as expected. I have avro data w/ schema registry, so I need to deserialize it. I followed the memgraph example here:…
Avocado
  • 871
  • 6
  • 23
2
votes
3 answers

Schema Registry Incompatible

How to check previous schemas registered in Registry? I'm trying to register but I am getting "Schema being registered is incompatible with an earlier schema for subject" exception
Gabriel Moraes
  • 21
  • 3
1 2 3
…
71 72