-1

This is a general question to understand the microsoft test software and its usage. Currently I am working with Microsoft Test Manager to create and run my manuall tests. In the project we have an Team Foundation Server which contains the project and we work on the code over visual studio. I don't know what team foundation server all contains. The unit tests are executed locally and seems not do be tracked by TFS. We work on a software that runs just on one computer and has only communication with another computer if this is required. So no web application or somethink like that

Now I would like to create automated UI tests and track them and the unit tests in TFS. Could please someone explain me what I have to do to accomplish this?

I have read that I need a lab environment to do automated tests. And for the environment I need additional software like test controller and test agent. But I could not install and get them to work.

Do I need to install the Test Controller on the TFS or where? Is Test Controller an own server software, included in TFS or like an Add on? Do every machine where I want to run and track test have the Test Agent software installed and configured?

I really hope someone can help me out.

Best Regards.

3 Answers3

2

Test Automation in TFS and Visual Studio is a broad subject. There is a sort of natural progression for test Automation.

  1. In Microsoft Test Manager you can do action recordings and use Deploying and testing web applications using Release Management to reduce the time spent on manual execution.

  2. Then you can import those Action recordings into Visual Studio to generate code from them. Without any further installation of tools you can run these with the Visual Studio Test runner on your machine.

  3. You can hand-code Unit tests and CodedUI tests using a Visual Studio Action Recorder. As with the tests generated from an Action Recording, you can play these back on your local machine without any further installation.

  4. If you want Test Manager or Visual Studio to gather additional data from your application under test, you must install the Visual Studio Test Agent on the system under test. You can configure these test agents using a .TestSettings file. This allows you to run the tests from your machine, but gather additional information.

  5. By associating your automated Tests to the Test Case work item and configuring Team Build for your application, MTM can execute the tests directly from MTM.

  6. If you want the tests to run the tests as part of your build, you're going to need a Test Environment. A Standard Test Environment requires at least one Test Controller and the Test Agent on the machine that run the test as well as each machine you want to monitor during test execution. These environments can be configured and used from both automated builds as well as from Visual Studio or MTM.

  7. If your development and test teams want to be able to quickly provision test labs using Hyper-V and virtualized labs, you can use the Test Agents in these environments as well to install and test your application.

Now, when you're ready to deploy your application to your non-development environments you can use Visual Studio Release Management to run your tests as part of the release management pipeline.

Since this is a very complex story, with all kinds of twists and turns I recommend you go through the Microsoft Patterns and Practices Guidance and the ALM Ranger guidance for testing. As well as some of the links referenced by Martin Hinshelwood.

So as you can see, the test agent and test controller are an essential element for certain scenarios, while its not needed for others.

The agent installers are available as a separate download from the Microsoft download site.

jessehouwing
  • 106,458
  • 22
  • 256
  • 341
0

All of the Microsoft testing integration is designed to be built into the automated build within TFS. So typically you have build controllers, and build agents installed which run the tests every time the build happens.

You can also install test controllers and agents if you do not want it to run at build time, or if you need more distributed test runs. (Load testing, etc)

If you just want to run ui tests you can still do it locally based off of test data, but you won't be able to track test results as integrated.

Tim
  • 2,878
  • 1
  • 14
  • 19
  • Sorry, This is totally inaccurate. – MrHinsh - Martin Hinshelwood Mar 09 '15 at 08:23
  • Sorry but no it's not. I have been using the tools for several years now and have a fully integrated build and test pipeline that performs all testing in an automated manner. If you have problems with some specific piece I said please use the suggest edit button, or otherwise say what you think is incorrect instead of simply saying that it is wrong. Maybe you don't use it this way, but it worked this way before there was a MTM so like I said, it was designed to be built in that manner. I never said that was the only way to use it. – Tim Mar 09 '15 at 11:41
  • All of Microsoft test integration is NOT built into build. There are many tools in that toolbox and you should use the right tool for the right job. – MrHinsh - Martin Hinshelwood Mar 10 '15 at 06:34
  • Test controllers and test agents are specifically designed for testing your application. Build is specifically designed to test your code. – MrHinsh - Martin Hinshelwood Mar 10 '15 at 06:35
  • Running UI should not be run locally and you can use data and track test results as integrated. – MrHinsh - Martin Hinshelwood Mar 10 '15 at 06:37
  • I said it was designed to be, not that it was the only way to be used. If manually triggered tests suite your process better, cool. Running the coded UI tests locally was one of the primary suggested ALM workflows, so yes it is fine in the case of desktop apps, and if you can still get automated, integrated results, cool I didn't realize that. The build-deploy-test pipeline is well documented in the ALM guides from MS, and allows the whole process to occur automatically on a regular basis, without any intervention. If you don't like using it that way it is certainly your right however. – Tim Mar 10 '15 at 10:40
  • The tool is designed as I suggested and I described the primary workflows. Build, deploy, test workflow is also as I described post 2012 and using build to deploy was a workaround pre 2013. Running codedUI tests is not, nor was ever designed to rub locally in any form other than to make sure that your test works. – MrHinsh - Martin Hinshelwood Mar 10 '15 at 16:24
  • I am glad you are now a MS engineer who knows how it was designed. From MSDN itself: Additional options for running coded UI tests: You can run coded UI tests directly from Visual Studio, as described earlier in this topic. Additionally, you can run automated UI tests from Microsoft Test Manager, or from Team Foundation Build. When coded UI tests are automated, they have to interact with the desktop when you run them, unlike other automated tests. And with that I am done. You cannot be reasoned with, nor do I care what you think. – Tim Mar 10 '15 at 18:04
  • Just because you can do a thing does bot mean that you should, or that it was designed, do a thing. I refer you to Sam Guckenheimer (the Product Owner of Visual Studio and TFS) book:http://www.amazon.co.uk/gp/aw/s/?ie=UTF8&k=Sam+Guckenheimer&i=books – MrHinsh - Martin Hinshelwood Mar 11 '15 at 16:34
  • If you are feeling a little more technical you could always read mine: http://www.amazon.co.uk/gp/aw/d/B00JDJDPJM/ref=mp_s_a_1_fkmr1_1?qid=1426091720&sr=8-1-fkmr1&pi=AC_SX110_SY165&keywords=professional+alm+with+vs+2013 – MrHinsh - Martin Hinshelwood Mar 11 '15 at 16:35
0

There are a number of different tools available for testing in TFS.

You can create and run manual tests using MTM and if you are collecting action recordings you can generate coded ui tests that can be associated with your test case in MTM. This will push results back to the test case when it is run as part of a test automation run.

If your coders are creating unit tests then they can be run on a build server. However you should only be running tests that do not need an instance of your application.

http://nakedalm.com/create-release-management-pipeline-professional-developers/

If you have instance tested, integration or UI, then you should create a release pipeline in the release management tooling that Congress with TFS.

http://nakedalm.com/execute-tests-release-management-visual-studio-2013/

Once you have a deployment you can run increasing levels of instance testing untill your application pops out the pipeline.

MrHinsh - Martin Hinshelwood
  • 23,409
  • 5
  • 46
  • 69