0

I had increased the partition of an existing Kafka topic via terraform. The partition size had increased successfully however when I test the connection to the topic, I'm getting a "Broker failed to validate record"

Testing method:

 echo "test" | kcat -b ...
**sensitive content has been removed**
...

% Auto-selecting Producer mode (use -P or -C to override)
% Delivery failed for message: Broker: Broker failed to validate record

I had tried to search up online and came across something called schema validation configuration: https://docs.confluent.io/cloud/current/sr/broker-side-schema-validation.html

Is there something I need to do after increasing the partition? ie flush some cache?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Elisha Choo
  • 1
  • 2

1 Answers1

0

You need to ask your Kafka cluster administrator if they have schema validation enabled, but increasing partitions shouldn't cause that. (This is a feature of Confluent Server, not Apache Kafka).

If someone changed the schema in the schema registry for your topic, or validation has suddenly been enabled, and you are sending a record from an "old" schema (or not correct schema), then the broker would "fail to validate" the record.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245