3

While writing some MSpec BDD tests I came across a scenario where a test that I was expecting to fail was passing, but only when I ran all of my tests. When I ran the test in isolation it failed as expected. After some investigation I found that some state that was set in a previous test was not being reset before the second test was running and this was causing the second test to pass when I expected it to fail. The following contrived code reproduces the scenario:

public class ContextBase
{
    protected static object state;
}

public class Context_a : ContextBase
{
    Establish context = () => { state = new object(); };

    It should_set_state = () => state.ShouldNotBeNull();
}

public class Context_b : ContextBase
{
    Establish context = () => {  };

    It should_set_state = () => state.ShouldNotBeNull();
}

Both of these test pass because Context_a is executed before Context_b and at the time Context_B is executed the state that was set in Context_A is still set. If you run Context_B in isolation then the test fails as the state has not been set. Interestingly if you remove the empty Establish statement from Context_B then Context_B will always fail as expected.

I'm relatively new to MSpec and this behaviour surprised me. I assumed that state like this would be reset between executing each context. Perhaps I have missed something... am I constructing these tests correctly? If MSpec does not reset state like this between contexts automatically then what strategy should I use to ensure that state is reset in cases like the one in my example? Should I put an Establish lambda on the ContextBase class that sets all state fields to null?

Anthony Mastrean
  • 21,850
  • 21
  • 110
  • 188
John
  • 2,715
  • 3
  • 17
  • 20

1 Answers1

2

MSpec doesn't "reset" static state between executing contexts. It obeys the behavior you know from normal static variables, i.e. they don't get reinitialized for the time the application (i.e. test run) is running unless you do that manually. It's best to initialize all fields in the Establish of each context.

Another option is to put an extra Establish on your base class, but that will hide important information from your context - you would have to navigate to the base class to see that a field was initialized with a certain value. But DRY doesn't apply to tests in general: I prefer to have base classes with protected static methods that I call from derived contexts (see this answer for an example).

Community
  • 1
  • 1
Alexander Groß
  • 10,200
  • 1
  • 30
  • 33
  • Thanks for the answer. I notice in the example you link to you also repeat the static state variables in each context instead of putting them on the base class, which I guess in itself also solves the issue I was having. Do you find that repeating code like this in each context complicates refactoring the tests when the design of the system changes? You mention that DRY doesn't apply to tests in general but I was always taught (perhaps wrongly) that I should take as much care to keep my tests DRY as I would with any other code. Thanks again for the answer! – John Feb 14 '13 at 12:21
  • Hey, John. To elaborate on Alexander's comment on DRY, executable specifications are intended to model a given interaction with a system. It's fine to factor out ancillary setup details, but factoring out collaborations that are pertinent to the understanding of a given interaction leads to obscure tests. For further reading, see: https://lostechies.com/derekgreer/2011/07/19/effective-tests-avoiding-context-obscurity/?preview=true&preview_id=514&preview_nonce=7b4ddbe974 – Derek Greer Jul 22 '15 at 19:32