How to interact with a window's GUI with Python?

Question

Let's say you want to open myapp.exe, open the 3rd menu, then choose the 2nd menu item (i.e. like a user would do with the keyboard or mouse), and then in the dialog window, choose the 2nd button.

pyahk and pyautogui seem to offer this, but in a rather "low-level" way, by simulating clicks:

pyautogui.click(x=moveToX, y=moveToY, clicks=num_of_clicks, interval=secs_between_clicks, button='left')

How to do interact with a Windows GUI in a higher-level way with Python?

Example:

window = gui.open('myapp.exe')
window.menu_open(3).choose_item(2)
child_window = window.wait_for_dialog()
child_window.buttons[1].click()

I do not believe such a method exists... You would have to attach a program to another program and be able to read and interpret what a "button" or "item" is despite the host program being written in an arbitrary language. As well many programs do not allow other programs to be attached to them because that is how you exploit and hack vulnerabilities in the host program. — Reedinationer, Feb 22 '19 at 18:43
@Reedinationer isn't there a unified way to browse through windows (list all the top-level `hWnd`? <- old memories from Winapi / C) — Basj, Feb 22 '19 at 18:47
Does [pywinauto](https://github.com/pywinauto/pywinauto) work how you want it? It seems like it's more high-level than the other libraries you mentioned. — Random Davis, Feb 22 '19 at 18:47
@RandomDavis Seems great! I think you can post it as an answer, or do you prefer I do it? — Basj, Feb 22 '19 at 18:49

score 12 · Accepted Answer · edited Feb 22 '19 at 18:50

12

pywinauto seems to be much more in-line with what you want - it utilizes the Win32 API and MS UI Automation among other things.

Here is an example of automation of the notepad application:

from pywinauto.application import Application
app = Application().start("notepad.exe")

app.UntitledNotepad.menu_select("Help->About Notepad")
app.AboutNotepad.OK.click()
app.UntitledNotepad.Edit.type_keys("pywinauto Works!", with_spaces = True)

edited Feb 22 '19 at 18:50

Basj

41,386
99
383
673

answered Feb 22 '19 at 18:49

Random Davis

6,662
4
14
24

1

It's exactly what I was looking for, amazing! – Basj Feb 22 '19 at 18:54
Can i also modify a currently running application rather than starting it? – jksevend Dec 31 '21 at 18:58
1

@jksevend absolutely. [here](https://pywinauto.readthedocs.io/en/latest/HowTo.html#how-to-specify-a-usable-application-instance) is how. – Random Davis Jan 03 '22 at 16:51
Great! thank you – jksevend Jan 04 '22 at 16:42

How to interact with a window's GUI with Python?

1 Answers1

Linked