AccessibleObjectFromPoint is returning client object instead of check box

Question

I am using Visual Studio 2017. I added two projects in a solution. One project is in C# with WPF. The other is in VC++ with ATL. From C#, I call a function in the VC++ project, which sets low level mouse hook. A part of the code in the low level mouse proc is as follows:

MSLLHOOKSTRUCT stMouse = *(MSLLHOOKSTRUCT*)(lParam);
POINT pt = stMouse.pt;
IAccessible* pAcc;
VARIANT varChild;
HRESULT hr = AccessibleObjectFromPoint(pt, &pAcc, &varChild);
VARIANT varRole;
hr = pAcc->get_accRole(varChild, &varRole);

When I test by clicking on a check box under View tab in MS Word 2013, then I get the role as client for WM_LBUTTONDOWN and WM_LBUTTONUP messages. But I should get the role as check box. I checked with Inspect.exe that comes with Windows 10 SDK. Inspect.exe shows role correctly as check box. Inspect.exe has two options - one to see UI Automation properties and the other to see MSAA properties. I am seeing the MSAA properties. How can I get the correct role? What method deos Inspect.exe uses?

Answer 1

Microsoft Active Accessibility / MSAA (based on the IAccessible interface) is a legacy API. You should now use Windows Automation. UIA is based on COM, and uses interfaces, the most important one being IUIAutomationElement. Inspect.exe uses UIA or MSAA.

Note .NET is compatible with UIA and WPF can expose its UI Elements to UIA pretty easily, through the AutomationPeer class and UIElement.OnCreateAutomationPeer Method method. WPF provides a default implementation that can be tweaked if needed.

Here is a similar example to yours in C++, using the UIA API, instead of MSAA (we could write the same using C#):

#include "stdafx.h" // needs <uiautomation.h>

class CCoInitialize { // https://blogs.msdn.microsoft.com/oldnewthing/20040520-00/?p=39243
public:
  CCoInitialize() : m_hr(CoInitialize(NULL)) { }
  ~CCoInitialize() { if (SUCCEEDED(m_hr)) CoUninitialize(); }
  operator HRESULT() const { return m_hr; }
  HRESULT m_hr;
};

// this is a poor-man COM object class
class CHandler : public IUIAutomationEventHandler
{
public:
  HRESULT QueryInterface(REFIID riid, LPVOID * ppv)
  {
    if (!ppv) return E_INVALIDARG;

    *ppv = NULL;
    if (riid == IID_IUnknown || riid == IID_IUIAutomationEventHandler)
    {
      *ppv = this;
      return NOERROR;
    }
    return E_NOINTERFACE;
  }
  ULONG AddRef() { return 1; }
  ULONG Release() { return 1; }

  // this will be called by UIA
  HRESULT HandleAutomationEvent(IUIAutomationElement *sender, EVENTID eventId)
  {
    wprintf(L"Event id: %u\n", eventId);
    return S_OK;
  }
};

int main()
{
  CCoInitialize init;

  // this sample uses Visual Studio's ATL smart pointers, but it's not mandatory at all
  CComPtr<IUIAutomation> uia;
  if (SUCCEEDED(uia.CoCreateInstance(CLSID_CUIAutomation)))
  {
    // get mouse pos now
    POINT pt;
    GetCursorPos(&pt);

    // find what type of "control" was under the mouse
    CComPtr<IUIAutomationElement> element;
    if (SUCCEEDED(uia->ElementFromPoint(pt, &element)))
    {
      CComBSTR type;
      element->get_CurrentLocalizedControlType(&type);
      wprintf(L"type at %u,%u: %s\n", pt.x, pt.y, type.m_str);
    }

    // get root
    CComPtr<IUIAutomationElement> root;
    if (SUCCEEDED(uia->GetRootElement(&root)))
    {
      // add a handler that will trigger every time you open any window on the desktop
      // in the real world, you'll need to delete this pointer to avoid memory leaks of course
      CHandler *pHandler = new CHandler();
      if (SUCCEEDED(uia->AddAutomationEventHandler(UIA_Window_WindowOpenedEventId, root, TreeScope::TreeScope_Children, NULL, pHandler)))
      {
        // since this is a console app, we need to run the Windows message loop otherwise, you'll never get any UIA events
        // for this sample, just press CTRL-C to stop the program. in the real world, you'll need to get out properly
        // if you have a standard windows app, the loop will be already there
        while (TRUE)
        {
          MSG msg;
          while (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
          {
            TranslateMessage(&msg);
            DispatchMessage(&msg);
          }
        }
      }
    }
  }
  return 0;
}