Controller Base not Working with Kafka Consumer in Background Service

Question

I am working on a Realtime Application where I have to consume messages from Kafka and process the message and create a status dictionary to display on webpage. The problem is that while Kafka is running as BackgroundService in my Application, The ControllerBase class is not working or say my app doesn't launch localhost:5000 or so.

using (var consumer = new ConsumerBuilder<string, string>(
                    (IEnumerable<KeyValuePair<string, string>>)configuration).Build())
                {
                    consumer.Subscribe(topic);

                    try
                    {
                        var message = consumer.Consume(cts.Token);
                        string consumedMessage = result.Message.Value.ToString();
                    }
                    catch (OperationCanceledException)
                    {
                        // Ctrl-C was pressed.
                    }
                    finally
                    {
                        consumer.Close();
                    }
                }
            }
            return Task.CompletedTask;
        }

Running this following service class in Background as soon as i comment out the consume part the localhost:5000 launches and if consume is present it doesn't.

1) You've only shown the consumer. Where is the web part of the code? 2) Have you tried using a Thread for the consumer? 2) Kafka isn't really meant for a request-reply blocking pattern like you seem to be doing — OneCricketeer, Jul 09 '22 at 18:19

score 0 · Answer 1 · answered Sep 30 '22 at 14:56

Your consumer needs to run in separate thread or it will block the main thread and thus the startup process.

Run your consumer in a function of the service, which you call asynchronously in the ExecuteAsync function.

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    await Task.Factory.StartNew(
        () =>
        {
            while (!stoppingToken.IsCancellationRequested)
            {
                YourConsumeFunction();
            }
        }
    );
}

Controller Base not Working with Kafka Consumer in Background Service

1 Answers1