2

I'm trying to write a NUnit 3 (3.8.1) extension that lets a test fail if it has a failed Debug.Assert(...) (instead of silently run through or even hang because it shows the failed assertion dialog).

In a NUnit 2 addin, I was able to so by unregistering all debug trace listeners and adding my own that just throws an exception (as for example explained here). However, this doesn't seem to work on NUnit 3 anymore.

I'm able to successfully deploy the extension and it's code is being executed.

[Extension(Description = "Failed Assertions Tracker", EngineVersion = "3.4")]
public class TrackerEventListener : ITestEventListener
{
    public void OnTestEvent(string report)
    {
        Console.WriteLine(report); // prints -> so I know this method is being called
        Debug.Listeners.Clear();
        Debug.Listeners.Add(new UnitTestTraceListener());
    }
}

However, my unit test unfortunately shows me that still the DefaultTraceListener is installed.

[Test]
public void FailingAssertionShouldNotHang()
{
    foreach (object listener in Debug.Listeners)
    {
        Console.WriteLine(listener.GetType().FullName);
    }
    Debug.Fail("I'm sorry. I've failed.");
}

And so the test shows the dialog instead of simply failing.

What am I doing wrong? I suspect that the call to the static Listeners collection is ineffective because the actual test is run in a different context (different AppDomain, process, ?). But if this is the case, how can I solve my problem?

Dejan
  • 9,150
  • 8
  • 69
  • 117
  • 1
    May be [this comment](https://stackoverflow.com/questions/2662041/can-i-configure-nunit-so-that-debug-fail-doesnt-show-a-message-box-when-i-run-m/2798663#comment61962711_3184631) will help – Aluan Haddad Dec 07 '17 at 01:31
  • @AluanHaddad: I don't want to repeat myself for every test fixture. Therefore, I want to write the logic as an extension. – Dejan Dec 07 '17 at 01:32
  • @AluanHaddad: sorry, I've read this too fast. Using SetUpFixtures (https://github.com/nunit/docs/wiki/SetUpFixture-Attribute) it seems possible to at least have the logic only once per tested assembly. That might be a fair-enough work-around. – Dejan Dec 07 '17 at 01:59
  • Using the `--inprocess` switch did not help. – Dejan Dec 07 '17 at 08:07

1 Answers1

1

It's important to keep in mind that NUnit 3 Extensions, while capable of replacing NUnit 2 Addins in a few cases, are actually entirely different technology. NUnit 3 Extensions extend the Engine, which is separate from the framework.

In this case, your extension is setting up a Trace Listener that will capture any Debug Trace or Assert output produced by the engine. If the engine contained Trace statements (it doesn't) you would get the output. Meanwhile, the framework is happily continuing to run tests on its own.

Any code that will successfully capture Trace has to be part of the actual framework execution of the tests. This gives you two options.

  1. Create a custom attribute that will capture trace. Custom attributes allow you to take actions when a test is being created or executed. They are created by implementing various interfaces supported by the framework. In your case, you would want to supply the attribute at assembly level, in order to capture all output produced by the assembly.

  2. Create the code as part of your tests themself, without extending the framework at all. You would want to capture the Trace output in an assembly-level SetUpFixture using the OneTimeSetUp attribute and release it under the OneTimeTearDown attribute. Since this approach is simpler than creating a custom attribute, it's the one I would use.

Charlie
  • 12,928
  • 1
  • 27
  • 31
  • I was secretly hoping that you'd answer my question ;) In that case, I'll try to implement 2. - Thank you for your continuous sharing and caring! – Dejan Dec 07 '17 at 23:02