Will Team Foundation Build Server execute UnitTests sequentially or in parallel

Question

We use TFS 2010 and Automated builds. We also make use of MSTests.

I would like some concrete information about the build server's test execution method. Will the test engine (on build server) run the unit tests sequentially or in parallel?

Answer 1

By default it will run them sequentially. You can customize the build workflow by adding a Parallel activity and running different sets of tests in each. Or if you want to parallelize the test run across multiple build machines you can have the build use multiple RunOnAgent activities to do so (http://blogs.msdn.com/b/jimlamb/archive/2010/09/14/parallelized-builds-with-tfs2010.aspx).

Note: If you execute the tests across multiple test runs you will end up with multiple test reports (.trx files) that will not be merged together without further customization of the build.

Answer 2

@Dylan Smiths answer is correct, but does not cover option # 3. Executing Unit Tests in parallel on a multi-CPU/core machine

DANGER WILL ROBINSON: This is only applicable to VS2010 and mstest.exe. VS2012 has a new test runner that does not support parallel test execution Visual Studio UserVoice Run unit tests in parallel The VS2012 test system can use the legacy testrunner, and you can make it work if you specify a .testsettings file using the MSTest/SettingsFile element. Configuring Unit Tests by using a .runsettings File

How to: Enable parallel test execution

1. Ensure you have a multi-core/CPU machine
2. Ensure you are running only unit tests
3. Ensure your tests are thread-safe
4. Ensure you do not have any data adapters on
5. Ensure you are running locally (cannot use TestController/TestAgent)
6. Modify your test settings file.
7. Right-click the test setting file and select "Open With" -> Open as Xml

8. Set the parallelTestCount attribute on the Execution element

   Options are:

   • blank = 1 CPU/Core - this is the default
   • 0 = Auto configure: We will use as many tests as we can based on your CPU and core count
   • n = The number of tests to run in parallel

9. Save your settings file