0

I have a simple batch file that calls MSBuild to build my Visual Studio solution, then runs the solution's NUnit tests, and then uses Wix to create an MSI. However I can't figure out how to stop the build process if any of the unit tests fail.

I tried the following:

nunit-console ../test.nunit
echo %ERRORLEVEL%

But regardless of whether unit tests passed or failed, %ERRORLEVEL% is always 0. Is there another way to easily determine whether any of the tests failed? I am using NUnit 2.5.9.

Phil
  • 6,561
  • 4
  • 44
  • 69
  • 1
    Since you're already using msbuild I would suggest using [Msbuild.Community.Tasks: NUnit Task](http://msbuildtasks.tigris.org/) it is more user friendly than batch. – Bas Bossink May 27 '11 at 17:40

2 Answers2

3

Call me lazy, but I just use MSBuildTasks and their NUnit Task. Then it's just a matter of setting the ContinueOnError property how you want it.

<NUnit ContinueOnError="true" Assemblies="@(TestAssemblies)" OutputXmlFile="$(TestResultsRoot)\nunit_results.xml" ToolPath="$(TeamBuildDependenciesPath)\NUnit\">
      <Output TaskParameter="ExitCode" PropertyName="NUnitResult" />
</NUnit>
Brook
  • 5,949
  • 3
  • 31
  • 45
  • This requires creating a whole new MSBuild script though, right? I'm trying to get MSBuild to just use my Visual Studio solution file to build everything. – Phil May 27 '11 at 17:46
  • Not really, I typically just edit up one of the .csproj files that VS creates. There's all kinds of steps in place which you can hook in to. It doesn't seem to mind. You can get fancy if you want to. – Brook May 27 '11 at 18:48
  • MSBuildTasks might be the least documented thing on the internet. How on earth do you use it? – Wouter Schut May 28 '15 at 19:19
0

After a while of searching it turns out the problem was that somewhere in my development process, the ERRORLEVEL variable was assigned a value. Thus when I wrote echo %ERRORLEVEL% it was reporting the ERRORLEVEL variable, not the actual ERRORLEVEL exit code.

I fixed this simply by closing the console and reopening it. Now when unit tests fail, the exit code is greater than 0. If all tests pass, the exit code is equal to 0.

EDIT: I completely removed this from my script. Now instead I put nunit-console $(TargetPath) in my test project's post-build events. That way, when I build the solution with MSBuild, it will automatically run unit tests and report an error when tests fail. When building in Visual Studio, this approach also causes an error to be added to the error list when unit tests fail, which is nice for identifying new bugs even more quickly.

Phil
  • 6,561
  • 4
  • 44
  • 69
  • problem is nunit-console returns errorlevel>0 for ignored tests – enorl76 Mar 14 '13 at 18:18
  • @enorl76 Then those tests shouldn't have run. It's possible to exclude tests from a certain category. An example of the exclude switch: `nunit-console myassembly.dll /exclude:Database` from the docs at https://www.nunit.org/index.php?p=consoleCommandLine&r=2.2 – T_D May 22 '17 at 09:52