2

I need to get the text from UI elements on a mac GUI.

One possible route might through pyobjc and UIAccessibility?

https://developer.apple.com/documentation/objectivec/nsobject/uiaccessibility https://pyobjc.readthedocs.io/en/latest/core/intro.html

I've RTFM for hours and can't figure it out. It doesn't help that I'm not a pyobjc or mac developer. I assume it's something so easy, there's no example for it online that I can find.

Maybe we can add a good example as the answer to this question :)

SwimBikeRun
  • 4,192
  • 11
  • 49
  • 85
  • 1
    When you say "text from UI elements", could you be a bit more specific or give concrete examples? Are you e.g. looking to control buttons a certain app (say SystemPreferences, Mail) or get the content of a TextEdit document? – Asmus Jul 24 '20 at 09:17

2 Answers2

0

From what I remember from previous tries, pyatomac proved pretty useful. They have a quite good documentation.

halfer
  • 19,824
  • 17
  • 99
  • 186
wschopohl
  • 1,567
  • 1
  • 11
  • 18
0

There is some information you can access about the windows and titles using the Python Quartz package, but it sounds like you want items rendered in the window. If it's just window info, use Quartz, here's the gist:

import Quartz
windows = Quartz.CGWindowListCopyWindowInfo( Quartz.kCGWindowListOptionAll, Quartz.kCGNullWindowID)
print windows[0].valueForKey_(kCGWindowBounds)
...

If you want the items rendered in the window, you'll need to be able to modify the child program so it sets NSWindow.sharingType = .readOnly. In addition, you'll most likely need to have spawned the child process. After all that, you may be able obtain a handle to the NSWindow using the CGWindowID:

 import AppKit
 windowID = windows[0].valueForKey_(kCGWindowNumber)
 window = NSWindow.windowWithWindowNumber(windowID)
 print window.contentView.subViews[0]
 ...

and introspect the NSWindow.contentView to find your target subview of type NSTextField, etc.

I haven't tried the second part using NSWindow.windowWithWindowNumber, but I have used the Quartz package in other projects successfully.

Marc Lavergne
  • 1
  • 2
  • 2