How to create a single producer?

Question

I have a message list coming from API, I need to map the elements in this list, send them to the producer and read them one by one. Multiple users can send message lists at the same time. I need to map the lists in order and not read the other list before one list is completely finished. Since a new producer is created for each request in the current structure, I cannot read the incoming messages sequentially. How can I do this by creating a single producer?

public class MailController : BaseController
    {
        [HttpPost("fatura")]
        //[AllowAnonymous]
        public bool fatura([FromBody] List<MailMessage> mailMessages)
        {
            FaturaProducer.Produce(mailMessages);
            return default;
        }
}

public class FaturaProducer : FProducer<MailMessage>
    {
        public static Task Produce(List<MailMessage> data)
        {
           Produce(ConfigurationManager.KafkaSettings.Topics[Topics.FaturaKaydetViaTp].TopicName, data);
           return default;
        }
    }

public class FProducer<T> : ProducerBase2
    {
        private static readonly ObjectSerializer<T> serializer = new ObjectSerializer<T>();
        //public static IProducer<Null, T> producer = new ProducerBuilder<Null, T>(producerConfig).SetValueSerializer(serializer).Build();
        public static async Task<DeliveryResult<Null, T>> Produce(string topic,List<T> data)
        {
            try
            {
                LogManager.Logger.Debug("Producer initiating for {topic}", topic);

                using (var producer = new ProducerBuilder<Null, T>(producerConfig).SetValueSerializer(serializer).Build())
                {
                    LogManager.Logger.Debug("Producer initiated");

                    LogManager.Logger.Debug("Producing async");

                    try
                    {
                        foreach (var item in data)
                        {
                            producer.Poll(TimeSpan.FromSeconds(5));
                            await producer.ProduceAsync(topic, new Message<Null, T> { Value = item });
                        }
                       
                    }
                    catch (ProduceException<Null, string> ex)
                    {
                        LogManager.Logger.Fatal(ex, "Delivery failed: {reason}", ex.Error.Reason);
                        throw;
                    }
                }
            }
            catch (Exception ex)
            {
                LogManager.Logger.Fatal(ex, "Producer failed for {topic}", topic);
            }

            return null;
        }

    }

score 1 · Answer 1 · answered Jan 06 '22 at 14:14

1

Currently, you're looping over the list, calling your Produce function, and creating one producer for each and every message

Alternatively, make a new Produce method that accepts the whole list, then wrap your foreach loop with using (var producer block so that only one producer is used for all messages, keeping in mind that you should not return on the very first loop item. You're not doing anything with the DeliveryResult, anyway, so make the method be void.

answered Jan 06 '22 at 14:14

OneCricketeer

179,855
19
132
245

I tried to send a list to the producer, but every time I make a request, a new producer is created, how can I single it out? – Gülsen Keskin Jan 07 '22 at 13:01
I see nothing wrong with that. But if you must, you'd need to create a producer object at the very start of your application, as a static singleton, and self-manage it rather than with a using statement around only code sections that need it – OneCricketeer Jan 07 '22 at 14:36

How to create a single producer?

1 Answers1