2

I am trying to run automated coded UI tests using Microsoft Test Manager, but I am getting the following error message in the test run log entry:

"Error starting the test run: Build directory of the test run is not specified or does not exist"

I already have a build assigned to the test run.

And I don't see how it can't find the build directory as I'm attempting to launch the tests from the build machine.

Edit: Perhaps a fake build can be created that I can link the tests to? How can this be done?

Update: I added the test project to the build, and re-associated the test case with the ordered test list, then created another build. I ensured that the build was output to the correct location, C:\GatedCheckinBuild, and I made sure that folder both created my test project and the ordered test list.

But still I get the same error when attempting to run the tests. Any ideas?

Ciaran Gallagher
  • 3,895
  • 9
  • 53
  • 97
  • Did your build compile and output the binaries containing the tests? MTM needs to run the tests from the DLLs the build made that are saved to the build drop folder. – Ryan Riehle Aug 02 '12 at 19:49
  • 1
    No I thought it might make more sense to keep UI test projects and production code separate, so I have been building it separately. Would it be easier to add the coded UI test projects to the production build, or would it be just as easy to create a build for the UI test code? – Ciaran Gallagher Aug 02 '12 at 19:59

2 Answers2

1

Tests must be assigned/linked to builds that contain the test DLLs. Add your test projects to the solution they are testing so that they are built together.

Ryan Riehle
  • 954
  • 4
  • 13
  • 1
    I suppose it does make sense to have test projects attached to the production builds so that test results, defects etc. can be associated with the build. Thanks. The MSDN documentation failed to detail this regarding the automation of tests using Microsoft Test Manager. – Ciaran Gallagher Aug 02 '12 at 20:55
  • Hope it goes well. I answered this one using my phone and I plan to update it for clarity and to make it more complete for others from my computer when I get a chance soon. – Ryan Riehle Aug 02 '12 at 23:33
  • 1
    Okay, so I added the test project to the production solution and checked it in from my environment, during which it appeared to create a checkin build to verify that it worked. Then I just attempted to run the tests using Microsoft Test Manager, but I got the same error. The build I assigned appears to be the checkin build which should contain the test project - which contains the ordered test list that I have associated with the test case linked to the test suite. Any ideas? Our builds output to C:\GatedCheckinBuild, and I can physically see the test project and the test list contained here. – Ciaran Gallagher Aug 03 '12 at 12:57
  • What I just noticed that actually, a copy of the build code is what gets output to C:\GatedCheckinBuild, the build DLL's actually get output to C:\GatedBuilds, so perhaps this is why Microsoft Test Manager can't find the build directory? Perhaps it's looking in C:\GatedCheckinBuild, when the build DLL's are actually being output to the other folder. I examined the build definition and did not see the C:\GatedBuilds folder mentioned anywhere. – Ciaran Gallagher Aug 03 '12 at 14:47
  • Okay so I investigated further, and found that the TFS builds "working directory" is set to C:\GatedBuilds. Is this why MS Test Manager is not running the tests? I tried copying all the DLL's into the build agent folder specified in the build definition and attempted to run the tests - but this didn't work. – Ciaran Gallagher Aug 03 '12 at 15:06
  • I'm sorry I hven't added more detail yet as i'm on an active vacation :p You shouldn't have to copy anything - if the solution your build is building includes the test project then it should output then to the build output share (this is a UNC path that you specify in the build definition). By default, the build will look for any DLLs with the word 'test' in their name; you can change this in the build definition or you can just ensure they have the word 'test' in the name of your project. – Ryan Riehle Aug 06 '12 at 20:06
  • No problem. Is that even documented anywhere? I was actually able to get it working - oddly it only worked when I placed the test DLL inside a bin/debug folder within the build output directory. I found this out not because it was documented somewhere, but because Microsoft Test Manager threw an error saying that it couldn't find the test DLL from the bin/debug folder... The build directory for our product places all the DLL's inside a 'Binaries' folder, not underneath a bin/debug folder structure. – Ciaran Gallagher Aug 06 '12 at 22:01
  • I see now that in the build definition process template there is the option to run automated tests by looking for DLL's with 'test' in the name - but this seems to only apply to automated tests that are run as part of the build. My tests won't be part of the build - the build will run on its own, and the tests will run on their own later. – Ciaran Gallagher Aug 16 '12 at 15:27
  • Is it possible to have test project in a separate solution? We just copy test DLLs manually to the agent server and try to run associated test from MTM. Here is a related question: http://stackoverflow.com/questions/38977638/automated-tests-in-microsoft-test-manager-build-directory-of-the-test-run-eithe – Denis Molodtsov Aug 16 '16 at 14:37
0

Select appropriate 'build in Use' which contains your test dlls. Change test plan properties for 'Build in Use' and save.

san_v
  • 1