2

After migrating to ReactiveUI 6.5.0 my tests started failing. The behaviour of Resharper Test Runner is very strange. When I'm running tests one after another - the tests pass but if I run them with big bang approach - Run Unit Tests (we've got about 1k Unit Tests) some tests are failing. This is one of them:

        [Test]
        public void TestThat_CallingRun_CallsLogin()
        {
            //act
            _sut.Actions.Single().Command.Execute(null);

            //assert
            _controller.AssertWasCalled(x => x.Login());
        }

        public ViewModel(IWfController workflowController)
        {
            _workflowController = workflowController;

            _runProcessCommand = ReactiveCommand.Create(CanRunProcess());
            _runProcessCommand.Subscribe(RunImpl);
        }

        private void RunImpl(object obj)
        {
            _workflowController.Login();
        }

_sut.Actions.Single().Command returns _runProcessCommand.

This test was passing with ReactiveUI 4.5. Is it guaranteed to be executed synchronously? If not - what is the proper way to test it now? Why is there are different behaviour between running them one after another? I would be glad for any ideas.

Edit:

I've noticed that RunImpl is being invoked on a different thread (only when running multiple unit tests) and it doesn't change anything if I replace

_runProcessCommand.Subscribe(RunImpl);

with

_runProcessCommand.ObserveOn(RxApp.MainThreadScheduler).Subscribe(RunImpl)

RunImpl is still being invoked on a different thread so assert is made BEFORE command is executed.

Edit 2: This is the fix which I invoke in NUnit [SetUp] function. If I observe on Scheduler.Immediate it won't work as well. Any ideas why it's not set to Immediate by default when I run multiple tests? (It doesn't work for all the failing tests)

 RxApp.MainThreadScheduler = Scheduler.Immediate;
 RxApp.TaskpoolScheduler = TaskPoolScheduler.Default;

Edit: After deeper debugging into RX sources in WaitForDispatcherScheduler I've noticed that:

IScheduler attemptToCreateScheduler()
        {
            if (_innerScheduler != null) return _innerScheduler;
            try {
                _innerScheduler = _schedulerFactory();
                return _innerScheduler;
            } catch (Exception) {
                // NB: Dispatcher's not ready yet. Keep using CurrentThread
                return CurrentThreadScheduler.Instance;
            }
        }
 RxApp.MainThreadScheduler = new WaitForDispatcherScheduler(() => DispatcherScheduler.Current);

When invoking scheduler factory for single test it's throwing an exception and returning CurrentThreadScheduler.Instance. When running multiple tests Dispatcher.Current returns an instance. Can anyone explain me what is going here?

Simple reproducable test:

    [Test]
    public void Test1()
    {
         var disp =DispatcherScheduler.Current; // throws exception (System.InvalidOperationException : The current thread has no Dispatcher associated with it.) when         running single tests, not throwing exception when running multiple tests.
    }

Edit 2: I have found what is causing DispatcherScheduler.Current to return different values. One of the previous test is using DelegateCommand which inside calls CommandManager.InvalidateRequerySuggested(); which creates dispatcher and value is returned by attemptToCreateScheduler() causing the rest of the tests failing because they are not called on Immediate Scheduler but Dispatcher (what is it in unit tests? It doesn't behave like immediate scheduler)

Edit3:

This behavior I've experienced at work and I couldn't reproduce it at home. But I am sure that something must be wrong unless there is something I don't understand about Reactive. I think there is no point of including the entire project, this example is showing exactly that scheduler is dispatcher instead of current thread if you use that function. Please debug it and you'll notice a different behaviour between these two calls in attemptToCreateScheduler() function in WaitForDispatcherScheduler class. ReactiveUI is version 7.0.

  [Test]
        public void TestMethod1()
        {
            RxApp.MainThreadScheduler.Schedule<string>(null, (x, y) => Disposable.Empty);
            CommandManager.InvalidateRequerySuggested();
            RxApp.MainThreadScheduler.Schedule<string>(null, (x, y) => Disposable.Empty);
        }
MistyK
  • 6,055
  • 2
  • 42
  • 76
  • It shouldn't be using the dispatcher at all - you can see [in the source](https://github.com/reactiveui/ReactiveUI/blob/rxui6-master/ReactiveUI/RxApp.cs#L66) that it's supposed to detect you're in a unit test and use `CurrentThreadScheduler`. What does `ModeDetector.InUnitTestRunner()` return in your unit tests? – Charles Mager May 13 '16 at 14:23
  • False. The problem is calling CommandManager.InvalidateRequerySuggested(); which creates dispatcher which as a result when RaiseAndSetIfChanged is raised it calls that function and dispatcher is found. You can test that - make an unit test an invoke CommandManager.InvalidateRequerySuggested(); and then call RaiseAndSetIfChanged. If you put a breakpoint inside ScheduledSubject after calling OnNext it will call attemptToCreateScheduler() and return dispatcher – MistyK May 13 '16 at 14:54
  • @up it's true not false. – MistyK May 13 '16 at 22:41

1 Answers1

1

There is a mismatch between the asynchronous nature of executing a command and the synchronous test.

Under the covers Execute just runs ExecuteAsync().Subscribe(), so all this will do is start the command running, there's no guarantees that the command will have finished executing by the time it returns. The fact it does in certain tests is entirely dependent on the schedulers in use.

What you should be doing is asynchronously waiting for the command to finish before asserting any side effects:

[Test]
public async Task Test()
{
    // arrange

    await command.ExecuteAsync();

    // assert
}

As an aside, and as noted in the various comments, I don't think the intention is for MainThreadScheduler to use the dispatcher in the unit test scenario. It does appear this may be a bug. See this github issue for further detail.

Charles Mager
  • 25,735
  • 2
  • 35
  • 45
  • @Zbigniew you need to debug splat's mode detector to work out why it doesn't think you're running in a unit test runner. The fact that command creates a dispatcher isn't that important - `MainThreadScheduler` shouldn't have been set to `WaitForDispatcherScheduler` in the first place. – Charles Mager May 13 '16 at 21:02
  • Are you sure? This is the line from Registrations.cs #if !MONO && !NETFX_CORE RxApp.MainThreadScheduler = new WaitForDispatcherScheduler(() => DispatcherScheduler.Current); I don't understand why u are saying about splay. Mode.IsUnitTestRunner() returns true every time – MistyK May 13 '16 at 21:43
  • Yes. Read the first para of my answer and the linked code. It is supposed to be replaced with `CurrentThreadScheduler.Instance` when running in a unit test runner. – Charles Mager May 13 '16 at 21:46
  • Please look at line 94 in your sources.It's returning _UnitTestMainThreadScheduler which is WaitForDispatchScheduler which is DispatcherScheduler after that CommandBinding function – MistyK May 13 '16 at 21:53
  • Hang on, `IsInUnitTestRunner` returns true? You'd said it returned false when i first asked... which is what the first part of the answer is predicated on. It does look from the source that there's a subtle bug if your test runner only uses a single thread. I think the intention is that the unit test case uses `CurrentThreadScheduler`. – Charles Mager May 13 '16 at 22:06
  • Sorry mate. In the previous post I thought of true and wrote false that's my mistake. In the next post I was correct but you could've not seen it. This test is obviously using single thread but I suppose you thought of dispatcher thread. If there is a dispatcher in Unit Test then the behaviour is wrong, is that correct? – MistyK May 13 '16 at 22:11
  • I don't think the intention was to use the dispatcher in a unit test - hence the check and changing it to `CurrentThreadScheduler`. Usually, it's not available. Perhaps that's worth raising as an issue on their github repo. The rest of the answer still stands - you need to be waiting for the command to complete in your tests. – Charles Mager May 13 '16 at 22:14
  • Yes but in my case it's causing even more weird behavior. Some commands even if awaited were not called at all and for some of them it helped. Look like a threading issue but I didn't investigate it further cause I found that dispatcher difference. Thanks mate for help, I really appreciate that. I'll give you a bounty in 10hours because I can't do it yet – MistyK May 13 '16 at 22:18
  • 1
    I found a [related issue](https://github.com/reactiveui/ReactiveUI/issues/854) logged on github and added a further comment there . – Charles Mager May 13 '16 at 22:26
  • I must have missed that. It's exactly the same issue - the funny thing is - raised exactly year ago :) – MistyK May 13 '16 at 22:28