Deadlock testing with TestSchedulers, Rx and BlockingCollection

Question

I have the following class which basically subscribes to an int observable and multiplies the value by 2. For reality purposes I added a Thread.Sleep to simulate a heavy processing.

public class WorkingClass
{
    private BlockingCollection<int> _collection = new BlockingCollection<int>(1);

    public WorkingClass(IObservable<int> rawValues)
    {
        rawValues.Subscribe(x => _collection.Add(x));
    }

    public IObservable<int> ProcessedValues()
    {
        return Observable.Create<int>(observer =>
        {
            while (true)
            {
                int value;

                try
                {
                    value = _collection.Take();
                }
                catch (Exception ex)
                {
                    observer.OnError(ex);
                    break;
                }

                Thread.Sleep(1000); //Simulate long work
                observer.OnNext(value * 2);
            }

            return Disposable.Empty;
        });
    }
}

I'm having trouble testing it, in the following test I just want to assert that if the source stream emits the value 1 the SUT will emit the value 2:

[Test]
public void SimpleTest()
{
    var sourceValuesScheduler = new TestScheduler();
    var newThreadScheduler = new TestScheduler();

    var source = sourceValuesScheduler.CreateHotObservable(
         new Recorded<Notification<int>>(1000, Notification.CreateOnNext(1)));

    var sut = new WorkingClass(source);

    var observer = sourceValuesScheduler.CreateObserver<int>();

    sut.ProcessedValues()
        .SubscribeOn(newThreadScheduler) //The cold part (i.e, the while loop) of the ProcessedValues Observable should run in a different thread
        .Subscribe(observer);

    sourceValuesScheduler.AdvanceTo(1000);

    observer.Messages.AssertEqual(new Recorded<Notification<int>>(1000, Notification.CreateOnNext(2)));
}

If I run this test the assert fails because the newThreadScheduler was never started and consequently the ProcessedValues observable was never created. If I do this:

 sourceValuesScheduler.AdvanceTo(1000);
 newThreadScheduler.AdvanceTo(1000);

It doesn't work either because the newThreadScheduler uses the same Thread of the sourceValuesScheduler so the test will be hanging right after the processed value is emmited, at the line:

value = _collection.Take();

Is there a way we can have multiple TestSchedulers running on different threads? Otherwise how can I test classes like this?

If ever you use `return Disposable.Empty;` to end an `Observable.Create` then you are creating an observable that can display all sorts of concurrency issues. You should never return `return Disposable.Empty;`. — Enigmativity, Aug 25 '19 at 03:31
What happens when you try this code: `public IObservable ProcessedValues() => _collection.GetConsumingEnumerable().ToObservable();` ? — Enigmativity, Aug 25 '19 at 03:38
I didn't know we should not use Disposable.Empty, are you saying that we should never use Observable.Create when there's nothing to dispose? Your suggestion of using _collection.GetConsumingEnumerable().ToObservable() causes a deadlock, because the processing has to be done in a different thread. On my example I was using a second scheduler, however the accepted answer uses the Task.Factory.StartNew(...). The problem is that it doesn't work when the Thread.Sleep is replaced by an await Task.Delay — Eduardo Brites, Aug 26 '19 at 13:09
When you use `Disposable.Empty` then the observable will have ended before the subscription has completed. That's what causes deadlocks. It's a bad anti-pattern to return it. I'll have a rethink of my suggestion, but I can't do that now. — Enigmativity, Aug 26 '19 at 13:29

score 0 · Accepted Answer · answered Aug 21 '19 at 13:20

Take() blocks until there is an item to remove from the BlockingCollection<int> or you call CompleteAdding() on it.

Given your current implementation, the thread on which you subscribe to ProcessedValues() and execute the while loop will never finish.

You are supposed to consume the BlockingCollection<int> on a separate thread. You may for example create a consume Task when ProcessedValues() is called. Consider the following implementation which also disposes the BlockingCollection<int>:

public sealed class WorkingClass : IDisposable
{
    private BlockingCollection<int> _collection = new BlockingCollection<int>(1);
    private List<Task> _consumerTasks = new List<Task>();

    public WorkingClass(IObservable<int> rawValues)
    {
        rawValues.Subscribe(x => _collection.Add(x));
    }

    public IObservable<int> ProcessedValues()
    {
        return Observable.Create<int>(observer =>
        {
            _consumerTasks.Add(Task.Factory.StartNew(() => Consume(observer), TaskCreationOptions.LongRunning));
            return Disposable.Empty;
        });
    }

    private void Consume(IObserver<int> observer)
    {
        try
        {
            foreach (int value in _collection.GetConsumingEnumerable())
            {
                Thread.Sleep(1000); //Simulate long work
                observer.OnNext(value * 2);
            }
        }
        catch (Exception ex)
        {
            observer.OnError(ex);
        }
    }

    public void Dispose()
    {
        _collection.CompleteAdding();
        Task.WaitAll(_consumerTasks.ToArray());
        _collection.Dispose();
    }
}

It can be tested like using the following code:

var sourceValuesScheduler = new TestScheduler();

var source = sourceValuesScheduler.CreateHotObservable(
    new Recorded<Notification<int>>(1000, Notification.CreateOnNext(1)));

var observer = sourceValuesScheduler.CreateObserver<int>();

using (var sut = new WorkingClass(source))
{
    sourceValuesScheduler.AdvanceTo(1000); //add to collection
    sut.ProcessedValues().Subscribe(observer); //consume
} //...and wait until the loop exists

observer.Messages.AssertEqual(new Recorded<Notification<int>>(1000, Notification.CreateOnNext(2)));

Do you know how to make it work if instead of a 'Thread.Sleep(1000)' we have an 'await Task.Delay(1000)'. The changes would be: private async Task Consume(IObserver observer) and 'Task.Factory.StartNew(async () => await Consume(observer)...' — Eduardo Brites, Aug 22 '19 at 16:25
@EduardoBrites: Please ask a new question if you have another issue. — mm8, Aug 26 '19 at 11:17

Deadlock testing with TestSchedulers, Rx and BlockingCollection

1 Answers1