How to unit test that tasks are run synchronously

Question

In my code I have a method such as:

void PerformWork(List<Item> items)
{
    HostingEnvironment.QueueBackgroundWorkItem(async cancellationToken =>
    {
        foreach (var item in items)
        {
            await itemHandler.PerformIndividualWork(item);
        }
    });
}

Where Item is just a known model and itemHandler just does some work based off of the model (the ItemHandler class is defined in a separately maintained code base as nuget pkg I'd rather not modify). The purpose of this code is to have work done for a list of items in the background but synchronously.

As part of the work, I would like to create a unit test to verify that when this method is called, the items are handled synchronously. I'm pretty sure the issue can be simplified down to this:

await MyTask(1);
await MyTask(2);
Assert.IsTrue(/* MyTask with arg 1 was completed before MyTask with arg 2 */);

The first part of this code I can easily unit test is that the sequence is maintained. For example, using NSubstitute I can check method call order on the library code:

Received.InOrder(() =>
{
    itemHandler.PerformIndividualWork(Arg.Is<Item>(arg => arg.Name == "First item"));
    itemHandler.PerformIndividualWork(Arg.Is<Item>(arg => arg.Name == "Second item"));
    itemHandler.PerformIndividualWork(Arg.Is<Item>(arg => arg.Name == "Third item"));
});

But I'm not quite sure how to ensure that they aren't run in parallel. I've had several ideas which seem bad like mocking the library to have an artificial delay when PerformIndividualWork is called and then either checking a time elapsed on the whole background task being queued or checking the timestamps of the itemHandler received calls for a minimum time between the calls. For instance, if I have PerformIndividualWork mocked to delay 500 milliseconds and I'm expecting three items, then I could check elapsed time:

stopwatch.Start();
// I have an interface instead of directly calling HostingEnvironment, so I can access the task being queued here
backgroundTask.Invoke(...);
stopwatch.Stop();

Assert.IsTrue(stopwatch.ElapsedMilliseconds > 1500);

But that doesn't feels right and could lead to false positives. Perhaps the solution lies in modifying the code itself; however, I can't think of a way of meaningfully changing it to make this sort of unit test (testing tasks are run in order) possible. We'll definitely have system/integration testing to ensure the issue caused by asynchronous performance of the individual items doesn't happen, but I would like to hit testing here at this level as well.

Answer 1

Not sure if this is a good idea, but one approach could be to use an itemHandler that will detect when items are handled in parallel. Here is a quick and dirty example:

public class AssertSynchronousItemHandler : IItemHandler
{
    private volatile int concurrentWork = 0;
    public List<Item> Items = new List<Item>();

    public Task PerformIndividualWork(Item item) =>
        Task.Run(() => {
            var result = Interlocked.Increment(ref concurrentWork);
            if (result != 1) {
                throw new Exception($"Expected 1 work item running at a time, but got {result}");
            }

            Items.Add(item);

            var after = Interlocked.Decrement(ref concurrentWork);
            if (after != 0) {
                throw new Exception($"Expected 0 work items running once this item finished, but got {after}");
            }
        });
}

There are probably big problems with this, but the basic idea is to check how many items are already being handled when we enter the method, then decrement the counter and check there are still no other items being handled. With threading stuff I think it is very hard to make guarantees about things from tests alone, but with enough items processed this can give us a little confidence that it is working as expected:

[Fact]
public void Sample() {

    var handler = new AssertSynchronousItemHandler();
    var subject = new Subject(handler);
    var input = Enumerable.Range(0, 100).Select(x => new Item(x.ToString())).ToList();

    subject.PerformWork(input);

    // With the code from the question we don't have a way of detecting
    // when `PerformWork` finishes. If we can't change this we need to make
    // sure we wait "long enough". Yes this is yuck. :)
    Thread.Sleep(1000);
    Assert.Equal(input, handler.Items);
}

If I modify PerformWork to do things in parallel I get the test failing:

public void PerformWork2(List<Item> items) {
    Task.WhenAll(
        items.Select(item => itemHandler.PerformIndividualWork(item))
        ).Wait(2000);
}

// ---- System.Exception : Expected 1 work item running at a time, but got 4

That said, if it is very important to run synchronously and it is not apparent from glancing at the implementation with async/await then maybe it is worth using a more obviously synchronous design, like a queue serviced by only one thread, so that you're guaranteed synchronous execution by design and people won't inadvertently change it to async during refactoring (i.e. it is deliberately synchronous and documented that way).