0

I'm trying to automate a windows form application using vb.net. I do not have the source code for the program I would like to automate.

With Spy++, so far I've been using handles for buttons and have been clicking buttons without an issue. However, I've stumbled across a problem with toolstrip buttons and I've been struggling to get around it. These buttons are hidden or invisible (from what I've read) and they don't show up in spy++ so they don't seem to have a handle; I can't therefore use the handle of the button to click it because it doesn't exist.

To solve the problem I've had to move the mouse cursor to a specific screen location, reposition the window and kick-off a mouse click event. This approach isn't the best though as the mouse can't be used while the automation code is running. Can anyone suggest an alternative? I've been trawling the internet and this site for a solution for three days without success.

Any help would be appreciated.

Watki02
  • 4,696
  • 7
  • 34
  • 36
user2068871
  • 1
  • 1

2 Answers2

0

In general, it is better to arrange your code such that there is a separation between your toolstrip button (or really ANY button you ever use on the interface) and your business logic in the backend. If possible, create a "code seam" there by writing a function that will be called by the button click handler, even if it is as simple as

(In VB)

Private Sub btnSomeButton_Click(ByVal sender As System.Object, _
                                ByVal e As System.EventArgs) _
                                Handles btnSomeButton.Click
     BusinessLogicClass.DoSomeButtonWork()
End Sub

This way you can easily call the function in other functions, use them in automation scripts, write unit tests and mocks for them, and refactor in one place instead of many. This is called "decoupling". That is a truly superior solution, especially if you are planning to use the process in a "batch mode".

Watki02
  • 4,696
  • 7
  • 34
  • 36
  • Hi, thanks for replying. The application I'm trying to automate has the buttons added to the toolstrip in the usual way. The application also has comboboxes in the toolstrip but these appear in spy++ so I can use the handles for these items to automate with. The buttons on the toolstrip don't show in spy++ though. btw, the toolstrip displays as a parent window in spy++. I can capture the handles for the comboboxes using findwindowex and can capture the handle for the toolstrip using findwindow. – user2068871 Feb 13 '13 at 16:30
  • I don't know much about "spy++". I guess you'll have to get help from someone else if you insist on using that route. I added a tag for spy++ onto your question for you. – Watki02 Feb 13 '13 at 19:03
  • Please accept my apologies. I didn't mean to go on about spy++. I don't have the ability to change the code in the application I'm trying to automate. I'd be happy to accept any of your suggestions as I am still stumped as to how I can click on a button in toolstrip without using the mouse position and the window position. – user2068871 Feb 14 '13 at 09:09
  • That's a pretty important detail you left out :). – Watki02 Feb 14 '13 at 15:54
  • The only other idea I have is to use a different program. For screen/mouse manipulation, you could try [AutoHotkey](http://www.autohotkey.com/). – Watki02 Feb 14 '13 at 16:05
0

I've managed to solve the problem myself!!! I'm so happy... lol. I didn't try autohotkeys but the solution I have found is a good one, or at least I think it is. An excellent explanation is provided here... http://msdn.microsoft.com/en-us/magazine/cc163288.aspx

Anyway, the solution is that vb.net has a "UI Automation" library/reference that you can add to your project. I saw it mentioned a few times online but couldn't figure out how to import it (sorry I'm a vb.net/coding newby). Anyway, you add the "UI Automation" reference to your project then you import it. You can then use the library to find all of the controls on the screen. This includes all of the buttons in the toolstrip.

I've added a code sample below to demonstrate how I solved it. I'm sure it's ropey in some way and could probably be done better but it works. Let me know what you think about the code/solution.

In order to run the code you need to know the index of the button and the index of the toolstrip. If you go through debug mode you can increment the index manually and check the controlName until the correct button comes up. In the example below I had to find the parent element (the window), then the child element (the toolstrip) and then the children of the child (all of the buttons under the toolstrip).

Imports system.windows.automation
Imports system.eventargs

'RoutedEvenArgs has to exist as a class so I declare it here...

Public class RoutedEventArgs Inherits EventArgs

end class

'my form code is all under one class - I'll probably break it up better
' but at the moment this is how it is

Public Class form1

Private Sub Button1_Click_1(ByVal sender As System.Object, _
                            ByVal e As System.EventArgs) Handles Button1.Click

'click the button of hte automation front end and 
' call the findtreeviewdescendants procedure

    FindTreeViewDescendants()

End Sub

Private Sub FindTreeViewDescendants()

'define the desktop as the rootelement as everywindow is a child of this element

Dim aedesktop As AutomationElement = AutomationElement.RootElement

'create an automationelementcollection variable to store the buttons

Dim aebuttons As AutomationElementCollection

'find the screen using the screen title 

aeform = aedesktop.FindFirst(TreeScope.Children, New PropertyCondition _
(AutomationElement.NameProperty, "Single Stock View"))

'find all the child controls (this brings back all controls including the toolstrip)

aebuttons = aeform.FindAll(TreeScope.Children, New PropertyCondition _(AutomationElement.IsControlElementProperty, True))

'create an automationelement to store the button and get information out of it

Dim a As AutomationElement

'each button, in the collection, has an index (incidentally the index number corresponds with the order in  which the window loads each of the elements into the window), in this case the toolstrip is index 1 as it's in the header of the screen

a = aebuttons.Item(1)

'get the child elements of the toolstrip element (something interesting is that in this case there were 19 elements but when you use findwindowex you only get back 4)

aebuttons = a.FindAll(TreeScope.Children, New PropertyCondition(AutomationElement.IsControlElementProperty, True))

'again use the index of the button to pull back the element information

a = aebuttons.Item(11)

'create a stringbuilder to store information about the element

Dim elementInfoCompile = New StringBuilder()

'identify the controlname, which in my case is the tooltip tag of the button (thus solving the problem of how I can find a button with an image instead of text in it)

Dim controlName As String
If (a.Current.Name = "") Then
controlName = "Unnamed control"
Else
controlName = a.Current.Name
End If

'identify the autoidname - which in all cases seemed to be null - I've no idea why but this didn't matter anyway

Dim autoIdName As String

If (a.Current.AutomationId = "") Then
autoIdName = "No AutomationID"
Else
autoIdName = a.Current.AutomationId
End If

'invoke a click of the button

InvokeControl(a)

End Sub 'FindTreeViewDescendants

'the rest is self-explanatory....

Private Sub InvokeControl(ByVal targetControl As AutomationElement)
Dim invokePattern As InvokePattern = Nothing

Try
invokePattern = _
DirectCast(targetControl.GetCurrentPattern(invokePattern.Pattern),  _
InvokePattern)
Catch e As ElementNotEnabledException
' Object is not enabled. 
Return
Catch e As InvalidOperationException
' Object doesn't support the InvokePattern control pattern 
Return
End Try

invokePattern.Invoke()

End Sub 'InvokeControl

End Class
Watki02
  • 4,696
  • 7
  • 34
  • 36
user2068871
  • 1
  • 1