Getting list of underlying menu items via UIAutomation with Delphi

Question

I have been trying to get a list of menu sub-items from a standard Windows application using the UIAutomationCore library imported as a TLB into Delphi - i.e.

File -> New | Exit
Help -> About

I can get the application menu, and then the top-level items into a list (i.e. in the example above, 'File' and 'Help', but I cannot get a list of ANY controls that are under these menuitems. My code is as below - the FElement represents the actual menuitem I am checking.

The length of the collection returned by FindAll is always 0. I have tried expanding the menuitem prior to this code, but it seems to have no effect.

 UIAuto.CreateTrueCondition(condition);

 FItems := TObjectList<TAutomationMenuItem>.create;

 self.Expand;
 sleep(3000);

 // Find the elements
 self.FElement.FindAll(TreeScope_Descendants, condition, collection);

 collection.Get_Length(length);

 for count := 0 to length -1 do
 begin
   collection.GetElement(count, itemElement);
   itemElement.Get_CurrentControlType(retVal);

   if (retVal = UIA_MenuItemControlTypeId) then
   begin
     item := TAutomationMenuItem.Create(itemElement);
     FItems.Add(item);
   end;
 end;

I can see examples of this in C#, and they are not really doing anything different from the code above (as far as I can see)

Thanks in advance

Update : It looks very similar to this question

Update2 : In this example it is trying to do this for another Delphi application. However, if I try the same thing on notepad (for example), the same issue occurs.

Update3 : Using Inspect (and then using UI Automation), I have the following structure ...

Name = Exit Ancestors = File (menu) Form1 (pane)

I have also tried this after expanding the menu (file), and the same thing is happening (or not happening).

Answer 1

I think you have the following two issues:

1. The menus will not list the sub menu items unless the menu is expanded
2. If you're trying to automate your own application, you have to do it in a thread.

The following works for me:

// Careful: the code might not be 100% threadsafe, but should work for the purpose of demonstration
const
  UIA_MenuItemControlTypeId =   50011;
  UIA_ControlTypePropertyId =   30003;
  UIA_NamePropertyId    =   30005;
  UIA_ExpandCollapsePatternId   =   10005;

procedure TForm1.Button1Click(Sender: TObject);
begin
  TThread.CreateAnonymousThread(procedure begin CoInitializeEx(nil, 2); FindItems(true); CoUninitialize; end).Start;
end;

procedure TForm1.FindItems(Recurse: Boolean);
var
  UIAuto: TCUIAutomation;
  condition: IUIAutomationCondition;
  collection: IUIAutomationElementArray;
  Length: Integer;
  Count: Integer;
  itemElement: IUIAutomationElement;
  retVal: Integer;
  val: WideString;

  ExpandCollapsePattern: IUIAutomationExpandCollapsePattern;
  FElement: IUIAutomationElement;
begin
  UIAuto := TCUIAutomation.Create(nil);   

  UIAuto.CreateTrueCondition(condition);

  // Find the elements
  UIAuto.ElementFromHandle(Pointer(Handle), FElement);

  FElement.FindAll(TreeScope_Descendants, condition, collection);

  collection.Get_Length(length);

  for Count := 0 to length - 1 do
  begin
    collection.GetElement(Count, itemElement);
    itemElement.Get_CurrentControlType(retVal);

    if (retVal = UIA_MenuItemControlTypeId) then
    begin
      ItemElement.Get_CurrentName(val);
      TThread.Synchronize(nil,
        procedure
        begin
          memo1.lines.Add(val);
        end);

      itemElement.GetCurrentPattern(UIA_ExpandCollapsePatternId, IInterface(ExpandCollapsePattern));
      if Assigned(ExpandCollapsePattern) then
      begin
        ExpandCollapsePattern.Expand;
        if Recurse = True then
          FindItems(False);
      end;
    end;
  end;
  UIAuto.Free;
end;