Continuous job queue finish jobs before closure

Question

Using the following code, I’m using a .net core Background Service with a continuous job queue where inside the ExecuteAsync simulates a job been added to the queue (this could be collecting an order, producing order responses etc)

The background service:

public class Worker : BackgroundService
{

    public WorkerJobQueue orderQueue { get; set; }

    public override async Task StartAsync(CancellationToken cancellationToken)
    {
        Console.WriteLine("Sales_Order_Processor_Service Starting");
        
        orderQueue = new WorkerJobQueue();
        orderQueue.RegisterHandler<TestJob>(TestJobWorker.DoJob);

        await base.StartAsync(cancellationToken);
    }

    public override async Task StopAsync(CancellationToken cancellationToken)
    {
        Console.WriteLine("Sales_Order_Processor_Service Stopping");
        await orderQueue.EndQueue(cancellationToken);
        await base.StopAsync(cancellationToken);
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var i = 0;

        while (!stoppingToken.IsCancellationRequested)
        {
            Console.ReadLine();

            for (var j = 0; j < 50; j++)
            {
                var tmp = new TestJob { JobNumber = i };
                Console.WriteLine($"Adding job {tmp.JobNumber} to queue");
                await orderQueue.Enqueue(tmp);
                i++;
            }

            Console.WriteLine($"{orderQueue.GetNumberOfRemainingJobs()} Jobs in queue...");

        }
    }

}

The Worker Job Queue:

public class WorkerJobQueue
{

    private ActionBlock<IJob> _workerBlock;

    public WorkerJobQueue()
    {
 
    }

    public void RegisterHandler<T>(Action<T> handleAction) where T : IJob
    {
        Action<IJob> actionWrapper = (job) => handleAction((T)job);
        var executionDataflowBlockOptions = new ExecutionDataflowBlockOptions()
        {
            MaxDegreeOfParallelism = 5,
        };
        _workerBlock = new ActionBlock<IJob>((job) => actionWrapper(job), executionDataflowBlockOptions);
    }

    public async Task Enqueue(IJob job)
    {
        await _workerBlock.SendAsync(job);
    }

    public int GetNumberOfRemainingJobs()
    {
        return _workerBlock.InputCount;
    }

    public async Task EndQueue(CancellationToken stoppingToken)
    {
        await Task.WhenAll(_workerBlock.Completion, Task.Delay(Timeout.Infinite, stoppingToken));
    }
}

The job item:

    public class TestJob : IJob
{
    public int JobNumber { get; set; }

}

And just to simulate some work done by the job:

    public class TestJobWorker
{
   
    public static void DoJob(TestJob testJob)
    {
        var rnd = new Random();
        var ranNum = rnd.Next(10);

        Console.WriteLine($"Starting job {testJob.JobNumber} sleeping for {ranNum} seconds");
        System.Threading.Thread.Sleep(ranNum* 1000);
        Console.WriteLine($"Finished job {testJob.JobNumber}");
    }

}

The job queue works as it should add 50 jobs in to the queue on key press, however whenever the service is stopped/the console window is closed its not actually awaiting the job queue to complete and doesn’t actually go into the StopAsync function?

Have I miss understood that the function StopAsync is called on closure/stopping the service? Or is the logic in my queue actually incorrect on ending the queue?

score 0 · Answer 1 · answered Sep 14 '20 at 17:17

Turns out the StopAsync is called later than the hosted stop i used the following code to triggure on stop:

        private readonly IHostApplicationLifetime _hostApplicationLifetime;
        public Worker(IHostApplicationLifetime hostApplicationLifetime)
        {
            _hostApplicationLifetime = hostApplicationLifetime;
        }

        private void OnStopping()
        {
            orderQueue.EndQueue();
            logger.Debug("Sales_Order_Processor_Service Stopping");
        }

Continuous job queue finish jobs before closure

1 Answers1