Produce called with an IAsyncSerializer value serializer configured but an ISerializer is required when using Avro Serializer

Question

I am working with Kafka cluster and using Transactional Producer for atomic streaming (read-process-write).

                // Init Transactions
                _transactionalProducer.InitTransactions(DefaultTimeout);

                // Begin the transaction
                _transactionalProducer.BeginTransaction();

                // produce message to one or many topics
                var topic = Topics.MyTopic;
                _transactionalProducer.Produce(topic, consumeResult.Message);

I am using AvroSerializer since I publish messages with Schema.

Produce throws an exception:

"System.InvalidOperationException: Produce called with an IAsyncSerializer value serializer configured but an ISerializer is required.\r\n   at Confluent.Kafka.Producer`2.Produce(TopicPartition topicPartition, Message`2 message, Action`1 deliveryHandler)"

All examples I've seen for transactional producer use Produce method rather than ProduceAsync so not sure I can simply switch to ProduceAsync and assume that transactional produce will function correctly. Correct me if I'm wrong or help me find documentation.

Otherwise, I am not able to find AvroSerializer that is not Async, inheriting from ISerializer.

public class AvroSerializer<T> : IAsyncSerializer<T>

score 4 · Accepted Answer · answered Jul 21 '20 at 17:46

I didn't realize that there is AsSyncOverAsync method which I can use when creating the Serializer. This exists because Kafka Consumer is also still Sync and not Async.

For example:

new AvroSerializer<TValue>(schemaRegistryClient, serializerConfig).AsSyncOverAsync();

Here is Confluent documentation of that method.

        //
        // Summary:
        //     Create a sync serializer by wrapping an async one. For more information on the
        //     potential pitfalls in doing this, refer to Confluent.Kafka.SyncOverAsync.SyncOverAsyncSerializer`1.
        public static ISerializer<T> AsSyncOverAsync<T>(this IAsyncSerializer<T> asyncSerializer);

Produce called with an IAsyncSerializer value serializer configured but an ISerializer is required when using Avro Serializer

1 Answers1