2

I have an excellent build server with a huge amount of RAM and processors. Also I have a project with a huge amount of MsTest tests (95% of which are simple unit tests). It is so disappointing to see these tests are executed one by one for 15 minutes. So I am looking for ways to execute these tests in parallel.

As I know TeamCity doesn't have native support for this. Of course you can create Build Configuration for each group of tests and execute them in parallel... but it sucks in many ways.

Continous Testing tools have excellent highly parallelized test runners inside. The best option I came to is to use them as Team City test runner but I have no idea how to implement it. Is there any way?

SiberianGuy
  • 24,674
  • 56
  • 152
  • 266

1 Answers1

2

You do not have to use the built in TeamCity test runner steps if you do not want to. You can run your build and tests from a single MSBuild script using a command line test runner and then upload the results to TeamCity.

Whilst you lose the GUI in building up your builds the benefit of doing the build this way is that your build should run the same on your development machine the same as it does on the build server. You are also CI agnostic with the ability to run your build on any CI environment (within reason).

TeamCity XML Report Processing

TeamCity Importing XML Reports

Here is an example:

<Message Text="##teamcity[importData type='mstest' path='path to mstest output xml file']"/>
Bronumski
  • 14,009
  • 6
  • 49
  • 77
  • I think this solution means the msbuild log itself cannot pass back any details? Is that right, or is there a way to get tc to accept a custom xml made up for reporting about it? (Docs on xml do not reference msbuild as an available input.) – AnneTheAgile Feb 20 '13 at 18:03
  • @AnneTheAgile Correct TeamCity's does not use the MSBuild logs to show the test results although the logs can be viewed as part of TeamCity's build logs. Instead as part of the test run, depending on the MSBuild task or external assembly you use, you upload to TeamCity the resulting xml output. TeamCity will then use this to show the test run results. The MSBuild Extension pack has a MSTest runner http://msbuildextensionpack.codeplex.com. – Bronumski Feb 20 '13 at 18:13
  • you can pass the results of mstest back through build service messages, see:http://confluence.jetbrains.com/display/TCD7/Build+Script+Interaction+with+TeamCity#BuildScriptInteractionwithTeamCity-ImportingXMLReports – James Woolfenden Feb 21 '13 at 16:30
  • @Bronumski, [cannot @ James] thank you for your replies! The TCD7 url says "typeID can be one of the following", and that list omits msbuild. Thus I infer that to pass back results to TC's log, it can only be done by a test of some kind, not pure msbuild. Did I get that right? That's why I thought msbuild couldn't pass back data. You are clarifying that if perhaps I had a dummy test, then I could, right? It would have to run after the build. – AnneTheAgile Mar 13 '13 at 19:56
  • @AnneTheAgile Hi Anne your runner is msbuild but your import type is mstest which is what you should set your typeid to. I have updated teh example to show this. – Bronumski Mar 14 '13 at 10:02
  • I'm running an NUnit regression test suite asynchronously on a separate test server, using PowerShell remoting triggered from a TC configuration to download the build under test. I'm trying to report the results in a second configuration, using a ##importData service message from a remote PowerShell task on the test server, but TC says "[NUnit report watcher] No reports found for paths:/[NUnit report watcher] C:\BuildAgent\work\67b5146301225c7e\TestResults.xml" so it appears to searching locally. Any advice on how to upload the results file to the TC server? – Francis Norton Apr 19 '13 at 16:09