0

I'm running tests in Visual Studio using "Test Explorer" with NUnit and a .runsettings file (specified by choosing the option in the GUI "Select Settings File")

My settings file (called mytests.runsettings) is:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <RunConfiguration>
    <DisableAppDomain>True</DisableAppDomain>
  </RunConfiguration>
  <ForceListContent>true</ForceListContent>
  <NUnit>
    <DomainUsage>None</DomainUsage>
  </NUnit>
</RunSettings>

I have verified that it is loading this file (verified by adding the framework node and setting it to a fake version, which then results in an error).

But no matter what I do, it doesn't run without an AppDomain!

Running from the command line does work:

nunit3-console.exe --domain=None --inprocess MyTests.dll

What do i need to do to get it to use that setting in NUnit?

Don Rhummy
  • 24,730
  • 42
  • 175
  • 330

2 Answers2

2

I don't believe you can do that.

  1. There is no recognized DomainUsage element under NUnit in the runsettings file. DomainUsage is an internally used property, which will be honored if set. But you can't set it that way. Your DomainUsage element is simply being ignored.

  2. If the adapter received your DisableAppDomain setting, then it would set DomainUsage to None. However, I don't believe it is actually receiving it.

Point 2 requires some explanation. Note that I haven't worked on the adapter for a few years and I'm going from memory but here it is...

The DisableAppDomain setting was added to allow Visual Studio to force NUnit to try to run without using an AppDomain. The Test Explorer is supposed to set things up so that it's possible to run that way, i.e. by making sure everything is already available in the current domain.

In order to prevent misuse of the feature, I believe that Test Explorer always overrides any user-provided setting. Again, this is from memory of work that was done a few years ago, but it seems as if the results you are seeing validate it.

The rationale for this past decision was that Test Explorer is completely responsible for setting up the Process and AppDomain used to run the tests. The user has no way to impact that and neither does NUnit. Of course, when using the console runner, that's not the case - control is in the hands of the user.

Something else to investigate is why you feel the need to run without a test AppDomain being created. But that's probably another question. :-)

I'll ask some other folks who may have a better memory than I to look at this as well.

UPDATE:

@Terje, who maintains the adapter now, replied and confirmed that there is no way to set DomainUsage in the runsettings file or any other way we know of when running under the test adapter. The docs have been corrected to avoid the implicit suggestion that it's possible.

We believe, but have not confirmed experimentally, that TestExplorer creates it's own AppDomain whenever it uses this setting to suppress its creation by the test adapter.

Charlie
  • 12,928
  • 1
  • 27
  • 31
  • I was confused, because this document ( https://github.com/nunit/docs/wiki/Tips-And-Tricks ) says you *can* set the `DomainUsage` in a runsettings file. Can I set `DomainUsage` via an environment variable? Or via a project properties command? – Don Rhummy Sep 19 '19 at 13:06
  • It could be that it was possible at the time wrote those tips. If you look at the current code at https://github.com/nunit/nunit3-vs-adapter/blob/master/src/NUnitTestAdapter/AdapterSettings.cs#L217 you can see that the setting is ignored. I already asked Terje to comment on my answer. – Charlie Sep 19 '19 at 15:31
  • Thanks, I appreciate it and look forward to Terje's insight too. Do you know why the Visual Studio setting doesn't work though? Line 238 seems to indicate it should: https://github.com/nunit/nunit3-vs-adapter/blob/master/src/NUnitTestAdapter/AdapterSettings.cs#L238 – Don Rhummy Sep 19 '19 at 16:07
  • Someone would have to debug it to figure it out. However, the `.runsettings` file is not read directly, IIRC, but is mediated by Test Explorer, so anything is possible. – Charlie Sep 20 '19 at 15:48
  • About point 1: @charlie is correct that [currently] this element can not be set from the runsettings file. That should have been specified in the wiki page. I do wonder however if this has been so all the time. – Terje Sandstrøm Sep 23 '19 at 16:19
  • @charlie is correct that [currently] this element can not be set from the runsettings file. That should have been specified in the wiki page. I do wonder however if this has been so all the time. Setting the DisableAppDomain to true will set the DomainUsage to None. If you enable the verbosity (>4), then you see what the adapter gets from these settings, and the DisableAppDomain is being read correctly from the runsettings file. It is then passed down to the – Terje Sandstrøm Sep 23 '19 at 16:25
  • - engine/framework, and should have worked the same way as the console. – Terje Sandstrøm Sep 23 '19 at 16:26
  • @terje My memory is that the TestExplorer team used a pseudo-runsetting to tell NUnit not to create an AppDomain. They did that rather than implement some other channel for the adapter to use, which I believe would have been cleaner. I don't think that the setting really should be in the user docs, since it's presence implies it's usable. – Charlie Sep 23 '19 at 18:19
  • @Charlie I removed it from the docs now. I don't see any other settings that affects this. The only settings I see and remember that affects the appdomain is the global DisableAppDomain, and that one is handled – Terje Sandstrøm Sep 23 '19 at 19:43
  • @TerjeSandstrøm Thank you for the help. I will try to re-verify, but from our testing, setting `DisableAppDomain=true` did *not* work. We got different results from when we used the nunit console runner and set `--domain=None` – Don Rhummy Sep 26 '19 at 19:17
  • @DonRhummy Note the update to the answer. When you set `DisableAppDomain` to `true`, we believe that TestExplorer then creates the domain. That's the purpose of the setting from their point of view. – Charlie Sep 27 '19 at 07:03
  • @TerjeSandstrøm It seems, that VS2022 no longer supports `DisableAppDomain` [Configure unit tests by using a .runsettings file](https://learn.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2022) – Wilhelm Medetz Jan 24 '22 at 16:46
0

This answer is a follow up to the answer from @charlie above, just need some more space here. I've checked on the domains that are being created, based on whether the DisableAppDomain is set or not. When the disable app domain is not set, the appdomain is created by NUnit with applicationbase and friendly name as shown below: enter image description here

The NUnitCheckDomain is the test dll.

When the disable app domain is set, NUnit no longer sets its own domain, and you then see it runs under the testhost appdomain, which is the process that runs all tests: enter image description here

So this seems to work the way it should and can. Do you need it to run under some other app domain than one of these?

BTW: If I run the NUnitCheckDomain test assembly using the NUnit3 console, then it a) works when run directly b) crashes when used with the parameters you give above (domain=None and --inprocess) with not able to load NUnit.Framework. @charlie - Any reason this should not work the same using NUnit3-Console ?

Terje Sandstrøm
  • 1,979
  • 15
  • 14
  • Thanks for helping here. The issue is we're using IIS via `HostedWebCore`, which appears to expect to be only in the default AppDomain. This works correctly when using NUnit at the command line with no domain, but VS Test creates an AppDomain other than default and then causes this issue. Any ideas? – Don Rhummy Sep 30 '19 at 18:42