Get control by position on pywinauto

Question

I have a mouse position on the screen, for example (10, 10) and I want to translate that posistion to a control. How can I do that?

Example:

from pywinauto.application import Application

app = Application((backend="uia").start("notepad.exe")
dlg = app.top_window()
hardcoded_file_button_rec = dlg.File.rectangle() #<RECT L10, T10, R40, B40>

given_mouse_position = (10, 10)
found_file_button = search_by_position(app, given_mouse_position)

assert hardcoded_file_button_rec == found_file_button.rectangle()

Does pywinauto already has built-in function for that? On this question I found how to iterate over all controls in a window using pywinauto. So iterating over it, I can check if controls[i].rectancle().top == 10 and controls[i].rectancle().left == 10.

What is the right thing to do?

score 2 · Accepted Answer · answered Dec 04 '18 at 14:09

Method .from_point(x, y) is under development.

Workaround is here: issue #413.

As well as on StackOverflow: How to pass POINT structure to ElementFromPoint method in Python?

Clean code sample for any element:

from ctypes.wintypes import tagPOINT
import pywinauto

elem = pywinauto.uia_defines.IUIA().iuia.ElementFromPoint(tagPOINT(x, y))
element = pywinauto.uia_element_info.UIAElementInfo(elem)
wrapper = pywinauto.controls.uiawrapper.UIAWrapper(element)

wrapper is what you need probably since it's actionable object having methods like .invoke() etc.

Get control by position on pywinauto

1 Answers1