I am currently working on a project where I need to connect multiple micro-services with Kafka and to serializer/deserializing messages with protobufer. The services are written with python and use python package "protobuf" (confluent-kafka).
I am currently trying to figure out how to read data with the consumer from topic with schema A and then process it and produce the proceed data to a topic with schema B.
Let's say I have the service MyService and it consumes from topic A with schema A_Schema
message A_Schema {
blah blah = 1;
blah blah = 2;
........
}
after processing the message from topic A, MyService needs to produce the data to topic B with schema B_Schema
message B_Schema {
different_blah different_blah = 1;
different_blah different_blah = 2;
........
}