NUnit API And Running Tests Programmatically

Question

This is the first time I'm trying to hack into the NUnit interface, and am facing some issues.

This is what I am trying to achieve:

• Take a set of string inputs as code files and compile them into an assembly
• take the in-memory assembly and pass it to NUnit for identifying and running the tests in this code

This is what I have so far

/// <summary> Validates whether the code files are valid </summary>
public string Validate(string[] codefiles)
{
    // To avoid NullReferenceException within NUnit
    // don't know why this is needed!
    TestExecutionContext.CurrentContext.TestPackage
        = new TestPackage("PersonTest");

    var assembly = BuildAssembly(codefiles);
    TestSuite suite = new TestAssembly(assembly, "PersonTest");
    var result = suite.Run(new NullListener(), TestFilter.Empty);
    return result.IsSuccess ? string.Empty : result.Message;
}

/// <summary> Builds an assembly </summary>
private static Assembly BuildAssembly(string[] code)
{
    var compilerparams = new CompilerParameters(new[] {"nunit.framework.dll"})
    {
        GenerateExecutable = false,
        GenerateInMemory = true
    };
    var results = new CSharpCodeProvider()
        .CompileAssemblyFromSource(compilerparams, code);

    if (!results.Errors.HasErrors) return results.CompiledAssembly;

    throw new Exception(GetErrors(results));
}

These are the two strings I am trying to compile (by sending as string array with two elements into the validate method above)

private const string File1 = @"
    public class Person 
    {
        public string Name {get;set;}

        public Person(string name)
        {
            Name = name;
        }
    }";

private const string ProperInput = @"
    using NUnit.Framework;

    [TestFixture]
    public class PersonTest
    {
        [Test]
        public void CheckConstructor()
        {
            var me = new Person(""Roopesh Shenoy"");
            Assert.AreEqual(""Roopesh Shenoy"", me.Name);
        }
    }";

Problem:

The compilation happens fine - the assembly generated even has these two types. Also not passing the nunit.framework.dll into referenced assemblies gives compilation exception (when dynamically compining), so it means that the TestFixture/Test addributes are recognized as well.

However, when I debug the TestAssembly just shows test count as zero. So instead of showing test success or failure, it shows inconclusive and basically just doesn't do anything.

I tried going through the NUnit codebase, but haven't reached any good conclusion yet - most of their convenient methods expect the assembly to be on the file system, so I'm trying to figure out exactly how to do this. Any ideas?

Answer 1

NUnit does some AppDomain creation when it runs, I wonder if somehow doing all this dynamically breaks that and it can't see your tests due to missing dependencies?

Answer 2

Ah it works!

I was missing a couple of steps (where the fixtures were separately identified and added to the assembly) + initially a call to "InstallBuiltIns" method. It works now and this is how it looks:

//this is needed only once
CoreExtensions.Host.InstallBuiltins();

var assembly = BuildAssembly(codefiles.ToArray());

// to avoid NullReferenceException - don't know why this is needed!
TestExecutionContext.CurrentContext.TestPackage
    = new TestPackage(assembly.GetName().FullName);

var suite = GetTestSuiteFromAssembly(assembly);
return suite.Run(new NullListener(), TestFilter.Empty);

where:

/// <summary>
/// Converts a given assembly containing tests to a runnable TestSuite
/// </summary>
protected static TestSuite GetTestSuiteFromAssembly(Assembly assembly)
{
    var treeBuilder = new NamespaceTreeBuilder(
        new TestAssembly(assembly, assembly.GetName().FullName));
    treeBuilder.Add(GetFixtures(assembly));
    return treeBuilder.RootSuite;
}

/// <summary>
/// Creates a tree of fixtures and containing TestCases from the given assembly
/// </summary>
protected static IList GetFixtures(Assembly assembly)
{
    return assembly.GetTypes()
        .Where(TestFixtureBuilder.CanBuildFrom)
        .Select(TestFixtureBuilder.BuildFrom).ToList();
}

The BuildAssembly is same as before. The only reason I had to repeat some of the logic from NUnit was that for some reason these were private within the NUnit Codebase!