8

I am playing with NUnit 2.6.3 and I did this tests:

using NUnit.Framework;
using System;

namespace NUnit26Tests
{
    [TestFixture]
    public class RandomTests
    {
        [Test]
        public void RandomTest([Random(1, 100, 5)] int value)
        {
            Assert.IsTrue(true);
        }

        [Test]
        public void SuccessTests()
        {
            Assert.That(true, Is.True);
        }
    }
}

But most of the execution times (99%) RandomTest is not executing on Test Runner.

This is the output message window:

------ Discover test started ------
NUnit 1.0.0.0 discovering tests is started
NUnit 1.0.0.0 discovering test is finished
========== Discover test finished: 6 found (0:00:00,9970583) ==========
------ Run test started ------
NUnit 1.0.0.0 executing tests is started
Run started: C:\TestProjects\NUnit26Tests\NUnit26Tests\bin\Debug\NUnit26Tests.dll
NUnit 1.0.0.0 executing tests is finished
Test adapter sent back a result for an unknown test case. Ignoring result for 'RandomTest(92)'.
Test adapter sent back a result for an unknown test case. Ignoring result for 'RandomTest(38)'.
Test adapter sent back a result for an unknown test case. Ignoring result for 'RandomTest(69)'.
Test adapter sent back a result for an unknown test case. Ignoring result for 'RandomTest(96)'.
========== Run test finished: 2 run (0:00:09,271531) ==========

In this case only one of five RandomTest's was executed.

I have tested with runner Nuget Package and installing NUnit Runner extension, same result.

Any idea what is the problem ?

ferpega
  • 3,182
  • 7
  • 45
  • 65
  • 1
    Did you try to run it manually with the NUnit `.exe` runners (Console or GUI)? Do you get the same result? – Dio F Jan 14 '14 at 13:24
  • 1
    @DioF Good caught ! it runs fine on the NUnit GUI so the problem seems related to the Visual Studio runner or runner Nuget package I have tested too. – ferpega Jan 14 '14 at 13:50

1 Answers1

4

I was able to reproduce this behavior. This seems to be a bug within the NUnit framework and/or the Test Adapter.

My guess is, that the random values are drawn once before the tests are run (to display them) and once when run. The random values drawn will probably not match and so the test results may not be assigned, leading to the mentioned error message.

You could open a bug for this issue at the project's development site (https://launchpad.net/nunitv2), but they are heavily busy with the upcoming v3 release.

As a workaround for your issue I propose that you use static (random) values (not using the RandomAttribute) or draw random values within your test (not as a parameter):

[Test]
[TestCase(15)]
[TestCase(38)]
[TestCase(2)]
[TestCase(72)]
[TestCase(69)]
public void RandomTest(int value)
{
    Assert.IsTrue(true);
}

Update

There is a known issue for this on github.

Dio F
  • 2,458
  • 1
  • 22
  • 39
  • As tests by @DioF this seems a problem related to the runner. Your answer to generate random is a workaround but because I am trying to test NUnity functionality itself (attribute Random in the example) it is not valid for me but I'm going to mark as solved for future visitors workaround. – ferpega Jan 14 '14 at 13:54
  • 2
    worth noting, for anyone looking in the future, that a similar thing happens when you use DateTime.Now in your test cases; as that value changes – simonalexander2005 Mar 08 '18 at 12:38