2

I have a suite of NUnit tests, some of which fail intermittently, probably because of timing problems. I'd like to find these flaky unit tests. Is there a way to repeat each test multiple times without having to put a Repeat() attribute on each test? We routinely use the resharper and ncrunch runners, but also have access to the nunit gui and console runners.

Phillip Ngan
  • 15,482
  • 8
  • 63
  • 79
  • You may be doing too much in each unit test - they should only contain a single test for one single part of your system. Are you able to add the code for the suspect unit test to your question? – Piers Myers Oct 06 '14 at 10:58
  • Hi Piers. I'm happy that the tests are not overly complicated. More that, the tests inherently have a timing dependency built in (using DateTime calculations, and timers etc). They pass most of the time, and fail unpredictably and sporadically. – Phillip Ngan Oct 06 '14 at 19:12

1 Answers1

3

NUnit 3

In NUnit 3, you may use Retry attribute:

RetryAttribute is used on a test method to specify that it should be rerun if it fails, up to a maximum number of times.

Notes:

It is not currently possible to use RetryAttribute on a TestFixture or any other type of test suite. Only single tests may be repeated.

If a test has an unexpected exception, an error result is returned and it is not retried. Only assertion failures can trigger a retry. To convert an unexpected exception into an assertion failure, see the ThrowsConstraint.

NUnit 2

NUnit 2 doesn't support retries, but you may use NUnit-retry plug-in (NuGet, GitHub). An example of use:

private static int run = 0;

...

[Test]
[Retry(Times = 3, RequiredPassCount = 2)]
public void One_Failure_On_Three_Should_Pass()
{
    run++;

    if (run == 1)
    {
        Assert.Fail();
    }

    Assert.Pass();
}

See also

Dariusz Woźniak
  • 9,640
  • 6
  • 60
  • 73