Non-deterministic unit tests with Rx, Xamarin

Question

So I'm writing unit tests for one of my view models in Xamarin mobile app. The method I'm testing looks like this:

public async Task RefreshItems()
{
    var departamentsObservable = _dataService.GetDepartaments();

    departamentsObservable.ObserveOn(SynchronizationContext.Current).Subscribe(items =>
    {
        Departaments.ReplaceWithRange(items);
    });

    await departamentsObservable.FirstOrDefaultAsync();
}

_dataService.GetDepartaments(); method returns IObservable<IEnumerable<Departament>>.

I use Observable and Subscribe instead of simple method that returns Task<IEnumerable<Departament>> because in my case Observable will "return" twice (once with data from cache and other time with newly fetched data form web).

For testing I of course mock _dataService.GetDepartaments(); method like that:

public IObservable<IEnumerable<Departament>> GetDepartaments()
{
    return Observable.Return(MockData.Departaments);
}

So method returns mock data immediately.

And my test for RefreshItems method looks like that:

[Fact]
public async Task RefreshItemsTest()
{
    await _viewModel.RefreshItems();

    Assert.Equal(MockData.Departaments, _viewModel.Departaments, 
                 new DepartamentComparer());
}

The problem is that this test randomly fails (1 in 10 times approximately). Basically The Departaments collection in view model that should be updated when Observable "returns" is empty.

I should add that I'm using xUnit 2.1.0 test framework and xUnit console runner in Xamarin Studio.

EDIT: The Enigmativity's suggestion throws Sequence contains no elements exception only when running in test runner. Below is minimal woking example code to demonstrate the issue:

using System;
using System.Collections.ObjectModel;
using System.Threading.Tasks;
using System.Reactive.Linq;
using System.Threading;
using System.Collections.Generic;

namespace TestApp
{
    public class TestViewModel
    {
        public ObservableCollection<TestDepartament> Departaments { get; set; }

        private ITestDataService _dataService;

        public TestViewModel(ITestDataService dataService)
        {
            _dataService = dataService;
            Departaments = new ObservableCollection<TestDepartament>();
        }

        public async Task RefreshItems()
        {
            var facultiesObservable = _dataService.GetDepartaments();

            await facultiesObservable.ObserveOn(SynchronizationContext.Current).Do(items =>
            {
                Departaments.Clear();
                foreach(var item in items)
                    Departaments.Add(item);
            });
        }
    }

    public interface ITestDataService
    {
        IObservable<IEnumerable<TestDepartament>> GetDepartaments();
    }

    public class MockDataService : ITestDataService
    {
        public IObservable<IEnumerable<TestDepartament>> GetDepartaments()
        {
            return Observable.Return(TestMockData.Departaments);
        }
    }

    public static class TestMockData
    {
        public static List<TestDepartament> Departaments
        {
            get
            {
                var departaments = new List<TestDepartament>();

                for (int i = 0; i < 15; i++)
                {
                    departaments.Add(new TestDepartament
                    {
                        Name = $"Departament {i}",
                        ImageUrl = $"departament_{i}_image_url",
                        ContentUrl = $"departament_{i}_content_url",
                    });
                }

                return departaments;
            }
        }
    }

    public class TestDepartament
    {
        public string ContentUrl { get; set; }
        public string Name { get; set; }
        public string ImageUrl { get; set; }
    }
}

And this is xUnit test:

public class DepartamentsViewModelTests
{
    private readonly TestViewModel _viewModel;

    public DepartamentsViewModelTests()
    {
        var dataService = new MockDataService();
        _viewModel = new TestViewModel(dataService);
    }

    [Fact]
    public async Task RefreshItemsTest()
    {
        await _viewModel.RefreshItems();

        Assert.Equal(TestMockData.Departaments, _viewModel.Departaments);
    }
}

Answer 1

Is your intention to block some GUI while this Refresh takes place. By block I mean block user progress by showing a spinner or other non-deterministic progress bar? Or Do you want the user to be able to continue doing what they were doing, but be able to signal that there is some background process happening, and also signal when that process is done?

For the Former, I would think that the current signature you have is sensible i.e. you return a Task. However the implementation I think can be improved.

public async Task RefreshItems()
{
    var items = await _dataService.GetDepartaments()
        .Take(1)
        .ObserveOn(SynchronizationContext.Current)
        .ToTask();

    Departaments.ReplaceWithRange(items);
}

Note the Take(1). If you try to convert from IObservable<T> to Task<T>/Task, you only get the last value, or yield when the sequence is complete. Without the Take(1) we could just wait forever. However, I think you have a scenario that you are loading from a cache, so can get 0,1 or 2 OnNext calls. In this scenario, I don't know what waiting for the last value achieves? I also note the lack error handling.

I would probably in my own code do something like this FWIW

public void RefreshItems()
{
    Departments.Clear();
    _state = States.Processing();
    var items =  _dataService.GetDepartaments()
        .SubscribeOn(_schedulerProvider.Background)
        .ObserveOn(_schedulerProvider.Foreground)
        .Subscribe(
            item=> Departaments.Add(item),
            ex => _state = States.Faulted(ex),
            () => _state = States.Idle());
}

This allows

• the _states field/property to reflect what is currently going on (processing, idle or some error),
• items to be added to the Departments list as the arrive instead of in one big block,
• Doesn't mix Task and IObservable<T>
• Has a single place to dictate concurrency model. It appears that there is something that is switching threads in your program that isn't in your code, as you have an ObserveOn but no matching SubscribeOn.

Edit

Here is a style I would go about creating this viewmodel. I prefer not to mix Task and IObservable. I also favor using Schedulers over Synchronization contexts. I have added resource management so that the subscriptions don't overlap (if Refresh is called multiple times) and so that it can be cancelled when the ViewModel is finished with. Here it should be easy to test 0,1 or many values being yielded. It also allows testing for error scenarios (e.g. OnError via Timeouts, Network outages etc.).

I also just for fun added the Async State property so that external consumers can see that the ViewModel is currently processing something.

I do however overload the test to have many concerns. I probably wouldn't do this in practice, but I think it makes it easier to read for SO.

http://share.linqpad.net/67hmc2.linq

void Main()
{
    var schedulerProvider = new TestSchedulerProvider();

    var cachedData = Enumerable.Range(0,3).Select(i => new TestDepartament
    {
        Name = $"Departament {i}",
        ImageUrl = $"departament_{i}_image_url",
        ContentUrl = $"departament_{i}_content_url",
    }).ToArray();

    var liveData = Enumerable.Range(10, 5).Select(i => new TestDepartament
    {
        Name = $"Departament {i}",
        ImageUrl = $"departament_{i}_image_url",
        ContentUrl = $"departament_{i}_content_url",
    }).ToArray();

    var data = schedulerProvider.Background.CreateColdObservable<IEnumerable<TestDepartament>>(
        ReactiveTest.OnNext<IEnumerable<TestDepartament>>(100, cachedData),
        ReactiveTest.OnNext<IEnumerable<TestDepartament>>(3000, liveData),
        ReactiveTest.OnCompleted<IEnumerable<TestDepartament>>(3000));

    var dataService = Substitute.For<ITestDataService>();
    dataService.GetDepartaments().Returns(data);

    var viewModel = new TestViewModel(dataService, schedulerProvider);

    Assert.Equal(AsyncState.Idle, viewModel.State);

    viewModel.RefreshItems();
    Assert.Equal(AsyncState.Processing, viewModel.State);

    schedulerProvider.Background.AdvanceTo(110);
    schedulerProvider.Foreground.Start();

    Assert.Equal(cachedData, viewModel.Departments);

    schedulerProvider.Background.Start();
    schedulerProvider.Foreground.Start();

    Assert.Equal(liveData, viewModel.Departments);
    Assert.Equal(AsyncState.Idle, viewModel.State);
}

// Define other methods and classes here
public class TestViewModel : INotifyPropertyChanged, IDisposable
{
    private readonly ITestDataService _dataService;
    private readonly ISchedulerProvider _schedulerProvider;
    private readonly SerialDisposable _refreshSubscription = new SerialDisposable();
    private AsyncState _state = AsyncState.Idle;

    public ObservableCollection<TestDepartament> Departments { get;} = new ObservableCollection<UserQuery.TestDepartament>();
    public AsyncState State
    {
        get { return _state; }
        set
        {
            _state = value;
            OnPropertyChanged(nameof(State));
        }
    }


    public TestViewModel(ITestDataService dataService, ISchedulerProvider schedulerProvider)
    {
        _dataService = dataService;
        _schedulerProvider = schedulerProvider;
    }

    public void RefreshItems()
    {
        Departments.Clear();
        State = AsyncState.Processing;
        _refreshSubscription.Disposable = _dataService.GetDepartaments()
            .SubscribeOn(_schedulerProvider.Background)
            .ObserveOn(_schedulerProvider.Foreground)
            .Subscribe(
                items =>
                {
                    Departments.Clear();
                    foreach (var item in items)
                    {
                        Departments.Add(item);  
                    }
                },
                ex => State = AsyncState.Faulted(ex.Message),
                () => State = AsyncState.Idle);
    }

    #region INotifyPropertyChanged implementation

    public event PropertyChangedEventHandler PropertyChanged;

    private void OnPropertyChanged(string propertyName)
    {
        var handler = PropertyChanged;
        if (handler != null) handler(this, new PropertyChangedEventArgs(propertyName));
    }

    #endregion

    public void Dispose()
    {
        _refreshSubscription.Dispose();
    }
}

public interface ITestDataService
{
    IObservable<IEnumerable<TestDepartament>> GetDepartaments();
}

public interface ISchedulerProvider
{
    IScheduler Foreground { get;}
    IScheduler Background { get;}
}
public class TestSchedulerProvider : ISchedulerProvider
{
    public TestSchedulerProvider()
    {
        Foreground = new TestScheduler();
        Background = new TestScheduler();
    }
    IScheduler ISchedulerProvider.Foreground { get { return Foreground; } }
    IScheduler ISchedulerProvider.Background { get { return Background;} }
    public TestScheduler Foreground { get;}
    public TestScheduler Background { get;}
}
public sealed class AsyncState
{
    public static readonly AsyncState Idle = new AsyncState(false, null);
    public static readonly AsyncState Processing = new AsyncState(true, null);

    private AsyncState(bool isProcessing, string errorMessage)
    {
        IsProcessing = isProcessing;
        IsFaulted = string.IsNullOrEmpty(errorMessage);
        ErrorMessage = ErrorMessage;
    }

    public static AsyncState Faulted(string errorMessage)
    {
        if(string.IsNullOrEmpty(errorMessage))
            throw new ArgumentException();
        return new AsyncState(false, errorMessage);
    }

    public bool IsProcessing { get; }
    public bool IsFaulted { get; }
    public string ErrorMessage { get; }
}

public class TestDepartament
{
    public string ContentUrl { get; set; }
    public string Name { get; set; }
    public string ImageUrl { get; set; }
}

Answer 2

Your code is creating two subscriptions to the observable - sometimes departamentsObservable.ObserveOn(SynchronizationContext.Current).Subscribe(...) produces values before await departamentsObservable.FirstOrDefaultAsync() completes and sometimes it doesn't.

Two subscriptions means two independent times that the source is run through. When the await completes quickly then your other subscription doesn't get called (or is called after your Assert.Equal is called), and hence the values are not added to the list.

Try this instead:

public async Task RefreshItems()
{
    var departamentsObservable = _dataService.GetDepartaments();

    await departamentsObservable.ObserveOn(SynchronizationContext.Current).Do(items =>
    {
        Departaments.ReplaceWithRange(items);
    });
}

Now you only have one subscription that awaits the last value to be produced.

If you're only expecting one value, and the observable doesn't naturally end, then pop a .Take(1) at the end.

Answer 3

@lee-campbell 's answer is great but it is missing Xamarin Forms implementation of his ISchedulerProvider.

Here is what I use:

public sealed class SchedulerProvider : ISchedulerProvider
    {
        public IScheduler Foreground => new SynchronizationContextScheduler(SynchronizationContext.Current);
        public IScheduler Background => TaskPoolScheduler.Default;
    }