13

We're using the VS 2010 test runner (MSTest) for automated functional testing. When we run tests from Visual Studio, VS creates a process called QTAgent32.exe, and it runs the tests in that process.

We've found that when we do multiple test runs, MSTest will reuse the same QTAgent32 process - the process ID does not change. This is a problem for us, since the code we're testing is P/Invoking to an unmanaged DLL. The DLL needs to be initialized only once during the lifetime of the process. We have an [AssemblyInitialize] method, but that executes once per test run. If we perform multiple test runs, it will execute more than once in the same process.

Every time we do a test run, MSTest creates a new appdomain; but these appdomains are all in the same process.

So I'm wondering: is there a way to tell the Visual Studio test runner to use a new process every time we run tests? I looked at the ".testsettings" configuration but didn't see anything relevant.

svick
  • 236,525
  • 50
  • 385
  • 514
Richard Beier
  • 1,712
  • 20
  • 25
  • 2
    Just for my curiosity - is it possible that the unmanaged DLL is unloaded after each consecutive test with `FreeLibrary`? – Wiktor Zychla Nov 18 '11 at 22:10
  • Thanks Wiktor - good suggestion. Right now the code just statically references the unmanaged APIs, using [DllImport] attributes on C# extern function declarations. I assume .NET doesn't automatically free the library when the appdomain is torn down, since we are getting errors. I don't actually know how to call a dynamically-loaded DLL from C# code - can you call LoadLibrary() and GetProcAddress(), and then somehow cast the returned function pointer to a delegate type? Anyway, I'll look into it - thanks! I just found this which may help: http://www.codeproject.com/KB/cs/dyninvok.aspx – Richard Beier Nov 18 '11 at 22:39
  • I have some C++ code I'm testing and I can confirm that MSTest does *not* unload DLLs between tests. It also does not call global variable destructors, so that makes me think it's the same executable the whole time. – jrh May 03 '21 at 19:38

3 Answers3

6

dont know how far you want to go with it, but one solution could be to create your unit test host

http://technet.microsoft.com/fr-fr/query/bb166558

this link shows how to create adapters, also you could then launch a new process for evertest, create a piped communication and tear it down after the test.

I know MS itself uses a different host for running tests under moles

http://research.microsoft.com/en-us/projects/pex/molestutorial.pdf

np-hard
  • 5,725
  • 6
  • 52
  • 76
  • Thanks! I'll check this out. I didn't realize you could create a custom host. Right now we're looking to see if there is a different test runner (e.g. Gallio Icarus) which can run the tests in a new process each time. It's really only an issue while interactively developing and testing - on the build server, we have just a single test run in a single test process. – Richard Beier Nov 18 '11 at 22:34
1

I was able to get this working after reading Wiktor's comment about FreeLibrary().

I used this class created by Mike Stall, which provides wrappers around LoadLibrary, GetProcAddress, and FreeLibrary. That way, I can load the library once in each test run, call the necessary methods, and then free the library at the end of the test run.

Mike Stall's code uses Marshal.GetDelegateForFunctionPointer, which converts an unmanaged function pointer to a managed delegate type.

I had to replace the [DllImport] extern declarations with declarations for delegate types. So I converted this:

[DllImport("asesignal.dll")]
public static extern bool ASESDK_Initialize(string licenseCode);

to this:

public delegate bool ASESDK_Initialize(string licenseCode);

Mike Stall's code contained examples with generic delegates (Action<T> etc.). But I couldn't get that working, so I created my own delegate types.

I can load the DLL dynamically like this:

_ht = new UnmanagedLibrary(@"c:\windows\system32\asesignal.dll");

To call a function, I do this:

var function = _ht.GetUnmanagedFunction<ASESDK_Initialize>("ASESDK_Initialize");
function(licenseCode);

Thanks Wiktor and np-hard for your help!

Richard Beier
  • 1,712
  • 20
  • 25
0

VS 2013 and forward now has a setting for this under Test > Test Settings > Keep Test Execution Engine Running. Unchecking this selection will start up a new engine each run.

tgriffin
  • 483
  • 1
  • 5
  • 13
  • This did not seem to have an effect, it's still running all the tests in the same process for me in Visual Studio 2013 – jrh May 03 '21 at 19:37