0

when i tried to run Kafka without schema registry,

I got an error like

CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL is required.
Command [/usr/local/bin/dub ensure CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL] FAILED !

Is schema registry mandatory for setting up kafka connect? But I didn't find any words like this from the confluent official documentation.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Sarath Baiju
  • 226
  • 2
  • 9

1 Answers1

2

The Registry is only required if you're using Confluent Converters that do need it, for example their AvroConverter, ProtobufConverter, or JsonSchemaConverter; it is not required for Connect itself.

If you want to run Connect in a container with minimal dependencies, see my image here - https://github.com/OneCricketeer/apache-kafka-connect-docker

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • Schema Registry is required when you are serializing or deserializing schema-based records using a converter. https://docs.confluent.io/home/connect/userguide.html#sr-long https://docs.confluent.io/home/connect/userguide.html#configuring-key-and-value-converters – Sarath Baiju Aug 10 '21 at 17:32
  • 1
    As answered, that only applies for Confluent-based Converters. For example, you can use schemas with `org.apache.kafka.connect.json.JsonConverter` , which do not need or use the Schema Registry, or you can write/find other converters on Github like BlueApron's Protobuf one or [one called "registryless-avro"](https://stackoverflow.com/questions/52333230/use-avro-in-kafkaconnect-without-confluent-schema-registry) – OneCricketeer Aug 10 '21 at 17:50
  • How can I opt the registry out? It's not mentioned in https://docs.confluent.io/platform/current/installation/docker/config-reference.html. – emeraldhieu Feb 01 '23 at 16:08
  • 2
    @emeraldhieu I don't understand what you're asking. Simply don't run the Schema Registry container. The Kafka Connect container does not need it. Set `CONNECT_VALUE_CONVERTER` to some class other than the ones I listed – OneCricketeer Feb 01 '23 at 17:13