Create a poker table scanner

Question

I'm trying to build a table scanner for the software pokerstars (this is totally legal).

The software looks like this:

The two parts that are interesting for the software are highlighted in orange. On the left part are all off the table names, on the right part are the players names off the currently active table.

I will call the left part tableView, and the right part playerView for later on

On the tableView, you can select which table you would like to see. You then see all the players sitting at that table in the playerView. What I would like to retrieve is basically a 2-dimenstional array:

[
    {
        "table": "Aeria IV",
        "players": [
            "careu82",
            "Noir_D€sir",
            "Onyon666",
            "plz542",
            "pripri17",
            "yataaaaaaaa"
        ]
    },
    {
        "table": "Aase III"
            "players":[...]
    },
    ...
]

To achieve this, I imagined the following process that I would like to automate:

1. move up to the first table in tableView
2. wait till the player names of that table show up in playerView
3. retrieve the tableName in tableView and the playerNames in playerView
4. move down to the next table in tableView
5. Do (2.) unless we reached the end off tableView

The programm I made so far uses UIAutomation.

It parses all the open windows (Chrome, paint, Pokerstars software) to find a window where the title contains "Lobby". This is the role of the function forEachWindow.

For the window that contains Lobby, it will look into all the UI Elements forEachUIElement. I'm for now retrieving the childcontent views of the pokerstars window: iuia.controlViewWalker.GetFirstChildElement(starsWindow), storing their position and width and height using uiElement.CurrentBoundingRectangle and their className using uiElement.CurrentClassName.

I'm then creating white windows with the same width/position as the UIElements to see which element is where.

As you can see, the two interesting elements are of class PokerStarsListClass

The code that does all of this is here:

from PyQt4.QtGui import *
from PyQt4.QtCore import *
from PyQt4.QtWebKit import *
import sys ,comtypes
from comtypes import *
from comtypes.client import *

# Generate and import a wrapper module for the UI Automation type library,
# using UIAutomationCore.dll
comtypes.client.GetModule('UIAutomationCore.dll')
from comtypes.gen.UIAutomationClient import *

# Instantiate the CUIAutomation object and get the global IUIAutomation pointer:
iuia = CoCreateInstance(CUIAutomation._reg_clsid_,
                        interface=IUIAutomation,
                        clsctx=CLSCTX_INPROC_SERVER)

rawView=iuia.rawViewWalker
controlView=iuia.controlViewWalker


def forEachUIElement(content_element,searchedClass=""):
    list=[]
    uiElement=controlView.GetFirstChildElement(content_element)
    while(uiElement!=None):
        try:
            rectangle=uiElement.CurrentBoundingRectangle
            width=rectangle.right-rectangle.left
            height=rectangle.bottom-rectangle.top
            y= rectangle.top
            x= rectangle.left
            className=str(uiElement.CurrentClassName)
            print className
            if searchedClass in className:
                list.append([uiElement,x,y,width,height,className])
        except Exception:
            ''
        try:
            uiElement=controlView.GetNextSiblingElement(uiElement)
        except Exception:
            uiElement=None
    print 'finish'
    return list

def forEachWindow(root_element):
    window=rawView.GetFirstChildElement(root_element)
    while(window!=None):
        try:
            windowName=window.currentName
            if 'Lobby' in windowName:
                print 'found'+windowName
                uiElements=forEachUIElement(window)
        except Exception:
            ''
        try:
            window=rawView.GetNextSiblingElement(window)
        except Exception:
            window=None
    return uiElements


class WindowOverLayer(QWebView):
    def __init__(self,parent):
        super(WindowOverLayer, self).__init__()
        self.parent=parent
        self.show()
    def closeEvent(self,event):
        self.parent.quit()

desktop = iuia.getRootElement()
uiElements=forEachWindow(desktop)
app=QApplication(sys.argv)
windowOverLayers=[]
for uiElement in uiElements:
    content= str(uiElement[5])
    windowOverLayer= WindowOverLayer(app)
    windowOverLayer.resize(uiElement[3],uiElement[4])
    windowOverLayer.move(uiElement[1],uiElement[2])
    windowOverLayer.setHtml(content)
    windowOverLayers.append(windowOverLayer)

app.exec_()

Given that I can now select the UIElements that are of class PokerStarsListClass, and say the two elements are stored in tableView and playerView, how can I automate the process ?

Here are also more infos about tableView, using the content properties:

CurrentAcceleratorKey: 
CurrentAccessKey: 
CurrentAriaProperties: 
CurrentAriaRole: 
CurrentAutomationId: 
CurrentBoundingRectangle: <ctypes.wintypes.RECT object at 0x00000000037A2948>
CurrentClassName: PokerStarsListClass
CurrentControlType: 50033
CurrentControllerFor: <POINTER(IUIAutomationElementArray) ptr=0x9466250 at 37a2948>
CurrentCulture: 0
CurrentDescribedBy: <POINTER(IUIAutomationElementArray) ptr=0x9466370 at 37a2948>
CurrentFlowsTo: <POINTER(IUIAutomationElementArray) ptr=0x9466310 at 37a2948>
CurrentFrameworkId: Win32
CurrentHasKeyboardFocus: 0
CurrentHelpText: 
CurrentIsContentElement: 1
CurrentIsControlElement: 1
CurrentIsDataValidForForm: 0
CurrentIsEnabled: 1
CurrentIsKeyboardFocusable: 1
CurrentIsOffscreen: 0
CurrentIsPassword: 0
CurrentIsRequiredForForm: 0
CurrentItemStatus: 
CurrentItemType: 
CurrentLabeledBy: <POINTER(IUIAutomationElement) ptr=0x0 at 37a2948>
CurrentLocalizedControlType: pane
CurrentName: 
CurrentNativeWindowHandle: 1574328
CurrentOrientation: 0
CurrentProcessId: 11300
CurrentProviderDescription: [pid:6188,hwnd:0x1805B8 Main:Nested [pid:11300,hwnd:0x1805B8 Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]