Why would an assert on a unit test with HostType("Moles") pass when run individually, but fail when run with a group of tests?

Question

I recently got aboard the Pex & Moles bandwagon in order to test some logic with many elements that are static, non-virtual, sealed, etc. Recently, I've begun to see behavior I can't explain from a few of the tests.

A couple of the methods for an interface I stubbed out return void, so I set the stubs to delegates that update boolean variables to indicate that they've been called. Here's what I'm doing:

[TestMethod]
[HostType("Moles")]
    public void UnitTestX()
    {
        bool disposeCalled = false;
        bool getCalled = false;

        ClassX classX = new ClassX();
        var data = new SIClassXData
                       {
                           Dispose = () => disposeCalled = true,
                           Get = () => getCalled = true,
                           ClassXGet = () => classX
                       };

        MDataLayerFactory.CreateDataLayerObject(() => (IClassXData)data);

        Assert.IsTrue(disposeCalled);
        Assert.IsTrue(getCalled);
    }

For whatever reason, the asserts above succeed if I run this test alone. But if I run the test along with every other test in the assembly (using Visual Studio's "Run All Tests in Solution" capability), the first assert fails.

I'd like to know why this occurs, and what I need to change to resolve the issue.

Is DataLayerFactory a statefull service? In that case, test case could steer the system to a state where it fails -- but does not fail in isolation. — Peli, Oct 15 '10 at 03:29
@Peli DataLayerFactory is a static class with nothing that could hold state. — Scott Lawrence, Oct 15 '10 at 14:54
FYI - in general we don't track actively stackoverflow question/answers on Pex/Moles (maybe we should but we don't for now) — Peli, Nov 01 '10 at 14:36

score 1 · Answer 1 · answered Apr 17 '12 at 22:57

Could it just be a side-effect of the 'run all tests' using multiple threads to execute the tests? So, that Dispose() has not run by the time the Assert fires?

Try using a ManualResetEvent to block the test method until Dispose() has run? Something like;

public void UnitTestX()
{
    // use this to block the test thread until Dispose() is run
    ManualResetEvent mre = new ManualResetEvent(false);

    bool disposeCalled = false;
    bool getCalled = false;

    ClassX classX = new ClassX();
    var data = new SIClassXData
                   {
                       Dispose = () => { 
                          disposeCalled = true; 
                          mre.Set(); // release the test thread now
                       },
                       Get = () => getCalled = true,
                       ClassXGet = () => classX
                   };

    MDataLayerFactory.CreateDataLayerObject(() => (IClassXData)data);

    Assert.IsTrue(mre.WaitOne(1000)); // give it a second to fire
    Assert.IsTrue(disposeCalled);
    Assert.IsTrue(getCalled);
}

Why would an assert on a unit test with HostType("Moles") pass when run individually, but fail when run with a group of tests?

1 Answers1