1

I have downloaded and installed in a windows server 2016 a vs test agent, which was installed by a tool called Test Agent Configuration Tool. I then followed this guide to configure the agent to run tests that interact with desktop - I want to run CodedUI Tests for web projects from whithin my release pipeline.

This is the output from the Test Agent Configuration Tool enter image description here

When I run my release pipeline I get this error during the test phase:

Error calling Initialization method for test class Captura.PDV.Web.Test.CodedUI.CodedUITest1: Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: To run tests that interact with the desktop, you must set up the test agent to run as an interactive process. For more information, see "How to: Set Up Your Test Agent to Run Tests That Interact with the Desktop" (https://go.microsoft.com/fwlink/?LinkId=255012)

If you are running the tests as part of your team build, you must also set up the build agent to run as an interactive process. For more information, see "How to: Configure and Run Scheduled Tests After Building Your Application" (https://go.microsoft.com/fwlink/?LinkId=254735)

Here is my test task configuration:

enter image description here

Some notes and questions:

  • I can run the tests calling vstest.console.exe manually from within the server with success
  • My release pipeline runs on VSTS
  • I am not using Microsoft Test Manager - Is it really necessary?
  • Do I need to set up a test lab enviroment?
  • Do I need to have a controller for the agent?

I just want to run my tests in first place to see how it goes and then proceed to more advanced tests scenarios using MTM

Bruno Casarotti
  • 623
  • 8
  • 23

1 Answers1

0

You're mixed up between the old way of doing things and the new way.

  • You don't need to use MTM -- MTM is deprecated.
  • You don't need to set up a lab environment. Lab environments are deprecated.
  • You don't need special test controllers or agents.

All you should need to do in order to run UI tests is:

  1. Set up a build/release agent that runs interactively.
  2. Run the tests on that agent with the Visual Studio Test task with the "Test Mix Contains UI tests" check box checked.

That's it.

I'd also recommend against using Coded UI and instead use Selenium (or Appium for desktop applications); Microsoft is no longer investing in the Coded UI platform and recommends using Selenium as an alternative.

Daniel Mann
  • 57,011
  • 13
  • 100
  • 120
  • I downloaded an agent from `https://{your_organization}.visualstudio.com/_admin/_AgentPool` installed it on my machine (tried both starting a session for the agent at the login time and also running it manually with `.\run.cmd`)and I got the same error telling me that the agent must be in interactive mode. After configuring the task to run with "test mix contains ui tests" I am also getting this: `2018-07-26T20:06:15.4361906Z ##[error]Unhandled: Unexpected end of JSON input 2018-07-26T20:06:15.4372741Z ##[error]Unhandled: Command failed: cscript.exe //Nologo C:\vstsagent\A1\_work\_tasks\VS..` – Bruno Casarotti Jul 26 '18 at 20:15
  • "Microsoft is no longer investing in the Coded UI" any sources for that? I need to justify it here at my company, this would be helpful – Bruno Casarotti Jul 26 '18 at 22:33
  • https://blogs.msdn.microsoft.com/visualstudioalm/2016/03/10/visual-studio-team-services-testing-tools-roadmap/ – Daniel Mann Jul 26 '18 at 22:38
  • 1
    @BrunoCasarotti, The choice between Coded UI and Selenium is not as clear cut as Daniel's words above suggest. I recommend careful reading of the Microsoft blog to see how your needs may be affected. – AdrianHHH Jul 29 '18 at 09:29
  • @AdrianHHH could you point out those articles from Microsoft blog? Any recomendations? From the blog post that Daniel sent, I clearly see the advantage of switching from Coded UI to Selenium or any other tool more compatible with WebDrive technology. I am starting to implement functional tests now on my solution, so this would be a good time to switch if needed. Would you recommend me to stay with Coded UI knowing that? If so why? – Bruno Casarotti Jul 30 '18 at 23:54
  • @BrunoCasarotti I do not understand what you are asking. I thought the Microsoft blog was clear and I recommended that you read it carefully. What other articles are you referring to? – AdrianHHH Jul 31 '18 at 08:18