2

I'm working on an automated test suite for a large WPF application. I've created a wrapper library for Microsoft Windows Automation API (previously known as UI Automation Framework), and I use it in order to implement various sanity tests. My team would like to do Continuous Integration with these tests by running them every check-in. We're working with TFS (2012 I believe) and MSTest.

The problem with these tests is that they require the UI to be up and running to actually run the tests. I guess the idea is to copy the built binaries to a remote VM and run the tests with the Remote Agent, but I'm not entirely sure on where to begin.

Has anyone ever done this before and has any code / build workflow XAML examples for achieving something like this?

Tal Bronfer
  • 759
  • 5
  • 18

2 Answers2

2

You need to set up a standard environment on the test machines, which will install a test agent on them. The test agent then runs the tests for you, on your UI.

This should get you started: http://msdn.microsoft.com/en-us/library/ee390842.aspx

Andrew Clear
  • 7,910
  • 5
  • 27
  • 35
  • I tried this today. Unfortunately, our TFS server is 2010, while we're using VS 2012. Naturally this is normally not a problem, however, you cannot integrate with a 2010 Test Controller through VS 2012, and you can't integrate a 2012 Test Controller with TFS 2010. I'm going to try using an external CI solution such as TeamCity. – Tal Bronfer Jul 31 '13 at 17:30
1

For my CI server, I use TeamCity by JetBrains. With it, you can setup multiple build agents to connect to the TeamCity server quite easily.

Specifically with regard to testing using UI Automation, you will need to have an interactive desktop session on all of your build agents as you mentioned. To do so, I set them all to auto-login and I have a startup script that runs C:\BuildAgent\bin\agent.bat start so the TeamCity agent starts in the interactive session.

TeamCity hooks in quite nicely to TFS and should be able to run your tests easily. I use NUnit as my test runner, but I can't imagine it being terribly difficult to get running. Best of all, it is free for three build agents and up to 20 build configurations.

Hope this helps.

Levi
  • 839
  • 2
  • 7
  • 21
  • What about a regular VM (not a build agent vm). Like the test controller and test agent set-up? – IbrarMumtaz Sep 10 '14 at 09:40
  • Sorry, I'm not sure I understand the question. Are you asking if a VM can be the build agent? Then yes. That's the setup I have. – Levi Sep 11 '14 at 15:27
  • I managed to figure it out, our build agents push out changes into a newly created VM and I was pondering about the interactive desktop thing you mentioned. I got around this with using simple controller and agent scenario. So in the end our build agents do not carry out automated ui tests. The newly created VM will carry out the tests. – IbrarMumtaz Sep 12 '14 at 15:04
  • How do you typically deploy such an environment? Our build agents run as a service and corp IT/security doesn't really allow machines to be auto logged on. How /where do you run these agents interactively and make everyone happy? – Christopher Painter Jan 06 '15 at 22:12
  • We have our own VM farm that we manage inside our network, so that's where I set these up. They're on our own dev domain, not the main one. I know you can set services to be interactive, but I've never had luck with getting UI Automation to interact with applications in this manner, unfortunately. – Levi Jan 07 '15 at 02:50