1

I am facing interesting issue. I've created TeamCity build step that runs unit tests using NUnit3 console (via command line).

Build step settings:

Command executable: SomeRootFolder\Packages\NUnit.ConsoleRunner.3.4.1\tools\nunit3-console.exe Command parameters: SomeRootFolder\Solution\Project.sln --process=single --timeout=300000 --where "cat != Slow"

Build step starts to execute as expected:

> [01:42:29][Step 2/4] NUnit Console Runner 3.4.1  
> [01:42:29][Step 2/4] Copyright (C) 2016 Charlie Poole 
> [01:42:30][Step 2/4] Runtime Environment 
> [01:42:30][Step 2/4]    OS Version: Microsoft
>                         Windows NT 6.3.9600.0 
> [01:42:30][Step 2/4] CLR Version: 
> [01:42:30][Step 2/4]    4.0.30319.34014 
> [01:42:30][Step 2/4] Test Files
> [01:42:30][Step 2/4]    SomeRootFolder\Solution\Project.sln
> [01:42:30][Step 2/4]  
> [01:42:30][Step 2/4] Test Filters
> [01:42:30][Step 2/4]    Where: cat != Slow
> Execution of 1000+ tests
> [02:37:32][Step 2/4] Test Run Summary
> [02:37:32][Step 2/4]  Overall result: Failed
> [02:37:32][Step 2/4]  Test Count: 1337, Passed: 1289, Failed: 0, Inconclusive: 0, Skipped: 48
> [02:37:32][Step 2/4]  Skipped Tests - Ignored: 48, Explicit: 0, Other: 0
> [02:37:32][Step 2/4]   Start time: 2016-09-15 23:42:52Z
> [02:37:32][Step 2/4]     End time: 2016-09-16 00:37:26Z
> [02:37:32][Step 2/4]     Duration: 3273.646 seconds
> [02:37:32][Step 2/4] 
> [02:37:32][Step 2/4] Results (nunit3) saved as TestResult.xml
> [02:37:33][Step 2/4] Process exited with code -2
> [02:37:33][Step 2/4] Step Nunit tests - investigation - full build (Command Line) failed

As long as at least 1 test fails, execution exits with code 0 and everything is reported in Team City with no issue (build step is marked as failed, statistics about failed/passed tests are reported etc.)

Once all tests pass, build step exits with code -2 even though it seems, that everything was alright.

I've also tried to test just some separate parts (specific assemblies) and again, build step completed with no issue and it was green (exited with code 0).

Any ideas, what is going on? Only occurrence when this is happening is, when all tests pass and build step is executed with whole solution.

Kajiyama
  • 3,393
  • 8
  • 26
  • 38

1 Answers1

2

-2 is the exit code provided when an invalid assembly has been found.

By running on a .sln - NUnit has to work out which of the built assemblies are relevant NUnit test assemblies, and try to only run those. As the .sln and .proj files change format now and then, this isn't a perfect bit of functionality, sometimes files are missed.

Try listing all your test dlls sequentially on the command line, and see if that solves the problem. e.g.

Command parameters: a.dll b.dll c.dll --process=single --timeout=300000 --where "cat != Slow"

Alternatively, it should be possible to look at TestResult.xml and search for "assembly" components marked as "invalid". If you find that NUnit is trying to run a project type it should be ignoring, you could submit a bug report.

Chris
  • 5,882
  • 2
  • 32
  • 57