Multiple Long running IHostedSerivces or BackgroundService implementations

Question

I cant seem to find much on this issue. The problem I am having is I have the need to be running 2+ long running background services but only the first registered service's ExecuteAsync gets executed. I have tried implementing it via BackgroundService and putting the code in ExecuteAsync and I have tried implementing IHostedService directly and putting the long running code in StartAsync.

I think the issue is return Task.CompletedTask; is never called. For example I have two Kafka consumers implemented as BackgroundServices The code looks identical in both aside from the topic and an OnMessage methods

protected override Task ExecuteAsync(CancellationToken stoppingToken)
{
    var kafkaEndpoint = _kafkaConfig.Endpoint;

    var kafkaTopic = "PhoenixEventStore";

    var consumerConfig = new Dictionary<string, object>
    {
        { "group.id", "consumer1" },
        { "bootstrap.servers", kafkaEndpoint },
        { "auto.offset.reset", "earliest" }
    };

    using (var consumer = new Consumer<Null, string>(consumerConfig, null, new StringDeserializer(Encoding.UTF8)))
    {
        consumer.OnMessage += (obj, msg) =>
        {
            Log.Information($"Consumer1 Received {msg.Value}");
        };

        consumer.OnPartitionEOF += (_, end) =>
        {
            Log.Information($"Consumer1 Reached end of topic {end.Topic} partition {end.Partition}.");
        };

        consumer.OnError += (_, error) =>
        {
            Log.Error($"Consumer1 Error: {error}");
        };

        consumer.Subscribe(new List<string>() { kafkaTopic });

        while (!stoppingToken.IsCancellationRequested)
        {
            consumer.Poll(TimeSpan.FromSeconds(10));
        }

        //consider setting value that check whether the consumer has stopped polling.
    }
    return Task.CompletedTask;
}

Since both services are long running the Task.Complete is never hit. However if I comment out the while loop both services ExecuteAsync are hit instead of just the first one registered.

I have found a work around that seems to be working but wondering if anyone else has a better approach.

Basically I refactor the code to have the long running code run in a void method called StartConsumer then have my ExecuteAsync look like this

  protected override Task ExecuteAsync(CancellationToken stoppingToken)
  {
        Task.Run(() => StartConsumer(stoppingToken));
        return Task.CompletedTask;
  }

Both services are registered using

services.AddHostedService<MyHostedService1>
services.AddHostedService<MyHostedService2>

I had the same problem, and your solution worked. Thank you. — Rafał Ryszkowski, Nov 29 '18 at 11:35

score 0 · Answer 1 · answered Jan 26 '21 at 09:49

I have the same case and it works. The template is

 protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                await Foo()
            }
            catch (Exception ex)
            {
                _log.Error(ex.Message, exception: ex);
            }

            await Task.Delay(timeInMlSeconds);
        }
    }

Multiple Long running IHostedSerivces or BackgroundService implementations

1 Answers1