0

I have a suite of integration tests that I run nightly through TFS's build/test agent framework. When tests that are not data driven fail, then I can examine their Error message in MTM via Test | Analyze Test Runs. However if the test is a data driven test 

[DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", @"|DataDirectory|\DataFiles\Providers.csv", "Providers#csv", DataAccessMethod.Sequential)]

and the test fails, the Error message field is not even present in the test results. Neither the summary nor the detail for the individual test that failed. As shown in ID 120574 below:

enter image description here

Running the test locally does provide an error message in the test explorer of Visual Studio, and in the cases I've encountered there is a mixture of pass & fail (i.e. one of the data driven cases failed but not all). I'm assuming that MTM is not showing the message because there is an aggregate of results.

Is there a way to configure my test, MTM, or the build to show these error messages for data driven tests?

Sean B
  • 11,189
  • 3
  • 27
  • 40
  • Those details should be there in the .trx file (attached to test results). Did you check that? – digitguy Jul 04 '15 at 09:19
  • @digitguy, I did, and they are not there. – Sean B Jul 06 '15 at 17:18
  • 1
    The ErrorMessage property may not be there but trx will show you the exception and stack trace of why the test failed. I guess that should be sufficient. However, I agree TFS/MTM is not consistent in the way its shows errors and exceptions, and the information we look for is spread across different places viz. ErrorMessage field, Test Log and Trx file. – digitguy Jul 07 '15 at 12:08
  • @digitguy, I stand corrected, you are correct. I discovered the corresponding section and it did in fact have the stack trace. It has a as well but it appears MTM is not parsing it, due to it not finding a element directly descending the . Thanks! I'll see if I can file a bug with MSFT. – Sean B Jul 07 '15 at 14:58

1 Answers1

1

Adding my comment as an answer for whoever is looking for solution.

The .trx file should have most (almost all) details about the test failure. It will have the Error Message, Exception and Stacktrace, wherever available, containing information about why a test failed/aborted/timeout.

Just in case nothing shows up in trx file, do check the Test Log as it may have information about Agent-Controller connection issues or other general network issues which could lead to test failures or abort.

digitguy
  • 1,014
  • 2
  • 12
  • 29