using C# and UI Automation to grab contents of unknown control-type

Question

In the image below there is an area, which has an unknown (custom) class. That's not a Grid or a Table.

I need to be able:

• to select Rows in this area
• to grab a Value from each cell

The problem is since that's not a common type element - I have no idea how to google this problem or solve it myself. So far the code is following:

Process[] proc = Process.GetProcessesByName("programname");
AutomationElement window = AutomationElement.FromHandle(proc [0].MainWindowHandle);
PropertyCondition xEllist2 = new PropertyCondition(AutomationElement.ClassNameProperty, "CustomListClass", PropertyConditionFlags.IgnoreCase);
AutomationElement targetElement = window.FindFirst(TreeScope.Children, xEllist2);

I've already tried to threat this Area as a textbox, as a grid, as a combobox, but nothing solved my problem so far. Does anybody have any advice how to grab data from this area and iterate through rows?

EDIT: sorry I've made a wrong assumption. Actually, the header(column 1, column 2, column 3) and the "lower half" of this area are different control-types!!

Thanks to Wininspector I was able to dig more information regarding these control types:

• The header has following properties: HeaderControl 0x056407DC (90441692) Atom: #43288 0xFFFFFFFF (-1)
• and the lower half has these: ListControl 0x056408A4 (90441892) Atom: #43288 0x02A6FDA0 (44498336)

The code that I've showed earlier - retrieved the "List" element only, so here is the update:

Process[] proc = Process.GetProcessesByName("programname");
AutomationElement window = AutomationElement.FromHandle(proc [0].MainWindowHandle);
//getting the header
PropertyCondition xEllist3 = new PropertyCondition(AutomationElement.ClassNameProperty, "CustomHeaderClass", PropertyConditionFlags.IgnoreCase);
AutomationElement headerEl = XElAE.FindFirst(TreeScope.Children, xEllist3);
//getting the list
PropertyCondition xEllist2 = new PropertyCondition(AutomationElement.ClassNameProperty, "CustomListClass", PropertyConditionFlags.IgnoreCase);
AutomationElement targetElement = window.FindFirst(TreeScope.Children, xEllist2);

After giving it a further thought I've tried to get all column names:

AutomationElementCollection headerLines = headerEl.FindAll(TreeScope.Children, new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.HeaderItem));
string headertest = headerLines[0].GetCurrentPropertyValue(AutomationElement.NameProperty) as string;
textBox2.AppendText("Header 1: " + headertest + Environment.NewLine);

Unfortunately in debug mode element count in "headerLines" is 0 so the program throws an error.

Edit 2: Thanks to the answer below - I've installed Unmanaged UI Automation, which holds better possibilities than the default UIA. http://uiacomwrapper.codeplex.com/ How do you use the legacy pattern to grab data from unknown control-type?

if((bool)datagrid.GetCurrentPropertyValue(AutomationElementIdentifiers.IsLegacyIAccessiblePatternAvailableProperty))
{
var pattern = ((LegacyIAccessiblePattern)datagrid.GetCurrentPattern(LegacyIAccessiblePattern.Pattern));
var state = pattern.Current.State;
}

Edit 3. IUIAutoamtion approach (non-working as of now)

        _automation = new CUIAutomation();
        cacheRequest = _automation.CreateCacheRequest();
        cacheRequest.AddPattern(UiaConstants.UIA_LegacyIAccessiblePatternId);
        cacheRequest.AddProperty(UiaConstants.UIA_LegacyIAccessibleNamePropertyId);
        cacheRequest.TreeFilter = _automation.ContentViewCondition;
        trueCondition = _automation.CreateTrueCondition();


        Process[] ps = Process.GetProcessesByName("program");
        IntPtr hwnd = ps[0].MainWindowHandle;
        IUIAutomationElement elementMailAppWindow = _automation.ElementFromHandle(hwnd);


        List<IntPtr> ls = new List<IntPtr>();

        ls = GetChildWindows(hwnd);

        foreach (var child in ls)
        {
            IUIAutomationElement iuiae = _automation.ElementFromHandle(child);
            if (iuiae.CurrentClassName == "CustomListClass")
            {
                var outerArayOfStuff = iuiae.FindAllBuildCache(interop.UIAutomationCore.TreeScope.TreeScope_Children, trueCondition, cacheRequest.Clone());
                var outerArayOfStuff2 = iuiae.FindAll(interop.UIAutomationCore.TreeScope.TreeScope_Children, trueCondition);

                var countOuter = outerArayOfStuff.Length;
                var countOuter2 = outerArayOfStuff2.Length;

                var uiAutomationElement = outerArayOfStuff.GetElement(0); // error
                var uiAutomationElement2 = outerArayOfStuff2.GetElement(0); // error
    //...
    //I've erased what's followed next because the code isn't working even now..
              }
        }

The code was implemented thanks to this issue:

Read cell Items from data grid in SysListView32 of another application using C#

As the result:

• countOuter and countOuter2 lengths = 0
• impossible to select elements (rows from list)
• impossible to get ANY value
• nothing is working

Answer 1

You might want to try using the core UI automation classes. It requires that you import the dll to use it in C#. Add this to your pre-build event (or do it just once, etc):

"%PROGRAMFILES%\Microsoft SDKs\Windows\v7.0A\bin\tlbimp.exe" %windir%\system32\UIAutomationCore.dll /out:..\interop.UIAutomationCore.dll"

You can then use the IUIAutomationLegacyIAccessiblePattern.

Get the constants that you need for the calls from:

C:\Program Files\Microsoft SDKs\Windows\v7.1\Include\UIAutomationClient.h

I am able to read Infragistics Ultragrids this way.

If that is too painful, try using MSAA. I used this project as a starting point with MSAA before converting to all UIA Core: MSSA Sample Code

----- Edited on 6/25/12 ------

I would definitely say that finding the proper 'identifiers' is the most painful part of using the MS UIAutomation stuff. What has helped me very much is to create a simple form application that I can use as 'location recorder'. Essentially, all you need are two things:

• a way to hold focus even when you are off of your form's window Holding focus

• a call to ElementFromPoint() using the x,y coordinates of where the mouse is. There is an implementation of this in the CUIAutomation class.

I use the CTRL button to tell my app to grab the mouse coordinates (System.Windows.Forms.Cursor.Position). I then get the element from the point and recursively get the element's parent until I reach the the desktop.

        var desktop = auto.GetRootElement();
        var walker = GetRawTreeWalker();
        while (true)
        {
            element = walker.GetParentElement(element);
            if (auto.CompareElements(desktop, element) == 1){ break;}
        }

----- edit on 6/26/12 -----

Once you can recursively find automation identifiers and/or names, you can rather easily modify the code here: http://blog.functionalfun.net/2009/06/introduction-to-ui-automation-with.html to be used with the Core UI Automation classes. This will allow you to build up a string as you recurse which can be used to identify a control nested in an application with an XPath style syntax.