1

I want to do some automation using pywinauto and I'm stuck at reading DataGridView table. How do I convert it to readable table? I need to see the actual data in the table. The columns in the table are dynamic, you can add or remove the one you need. As far as I understand I need to convert ir using ListViewWrapper but I can't get the element of the table. The control identifiers:

app = Application(backend='uia').connect(title_re=app_regex)
dgv = app.top_window().DataGridView
dgv.print_control_identifiers()

Table - 'DataGridView'    (L198, T191, R1366, B621)
['DataGridViewTable', 'Table', 'DataGridView']
child_window(title="DataGridView", auto_id="dgvBrowser", control_type="Table")
   | 
   | ScrollBar - 'Vertical Scroll Bar'    (L1348, T192, R1365, B620)
   | ['ScrollBar', 'Vertical Scroll BarScrollBar', 'Vertical Scroll Bar']
   | child_window(title="Vertical Scroll Bar", auto_id="4261242", control_type="ScrollBar")
   |    | 
   |    | Button - 'Line up'    (L1348, T192, R1365, B209)
   |    | ['Button', 'Line up', 'Line upButton', 'Button0', 'Button1']
   |    | child_window(title="Line up", control_type="Button")
   |    | 
   |    | Thumb - 'Position'    (L1348, T209, R1365, B590)
   |    | ['Position', 'PositionThumb', 'Thumb']
   |    | child_window(title="Position", control_type="Thumb")
   |    | 
   |    | Button - 'Page down'    (L1348, T590, R1365, B603)
   |    | ['Button2', 'Page down', 'Page downButton']
   |    | child_window(title="Page down", control_type="Button")
   |    | 
   |    | Button - 'Line down'    (L1348, T603, R1365, B620)
   |    | ['Button3', 'Line down', 'Line downButton']
   |    | child_window(title="Line down", control_type="Button")
   | 
   | Custom - 'Top Row'    (L198, T191, R1348, B213)
   | ['Custom', 'Top RowCustom', 'Top Row', 'Custom0', 'Custom1']
   | child_window(title="Top Row", control_type="Custom")
   |    | 
   |    | Header - ''    (L199, T191, R239, B213)
   |    | ['', 'Header', '0', '1', 'Header0', 'Header1']
   |    | 
   |    | Header - ''    (L239, T191, R264, B213)
   |    | ['2', 'Header2']
   |    | 
   |    | Header - 'Data'    (L264, T191, R352, B213)
   |    | ['DataHeader', 'Header3', 'Data']
   |    | child_window(title="Data", control_type="Header")
   |    | 
   |    | Header - 'Technikas'    (L352, T191, R484, B213)
   |    | ['TechnikasHeader', 'Technikas', 'Header4']
   |    | child_window(title="Technikas", control_type="Header")
   |    | 
   |    | Header - 'Paskyros numeris'    (L484, T191, R638, B213)
   |    | ['Paskyros numerisHeader', 'Header5', 'Paskyros numeris']
   |    | child_window(title="Paskyros numeris", control_type="Header")
   |    | 
   |    | Header - 'Objektas-Adresas'    (L638, T191, R762, B213)
   |    | ['Objektas-Adresas', 'Objektas-AdresasHeader', 'Header6']
   |    | child_window(title="Objektas-Adresas", control_type="Header")
   |    | 
   |    | Header - 'Klientas'    (L762, T191, R866, B213)
   |    | ['Klientas', 'Header7', 'KlientasHeader']
   |    | child_window(title="Klientas", control_type="Header")
   |    | 
   |    | Header - 'Stadija'    (L866, T191, R997, B213)
   |    | ['StadijaHeader', 'Header8', 'Stadija']
   |    | child_window(title="Stadija", control_type="Header")
   |    | 
   |    | Header - 'Darbu vykdytojas'    (L997, T191, R1147, B213)
   |    | ['Darbu vykdytojasHeader', 'Darbu vykdytojas', 'Header9']
   |    | child_window(title="Darbu vykdytojas", control_type="Header")
   |    | 
   |    | Header - 'Projektas'    (L1147, T191, R1348, B213)
   |    | ['ProjektasHeader', 'Header10', 'Projektas']
   |    | child_window(title="Projektas", control_type="Header")
   | 
   | Custom - 'Row 0'    (L199, T213, R1348, B234)
   | ['Custom2', 'Row 0Custom', 'Row 0']
   | child_window(title="Row 0", control_type="Custom")
   |    | 
   |    | Custom - ' Row 0'    (L199, T213, R239, B234)
   |    | ['Custom3', ' Row 0', ' Row 0Custom', ' Row 00', ' Row 01', ' Row 0Custom0', ' Row 0Custom1']
   |    | child_window(title=" Row 0", control_type="Custom")
   |    | 
   |    | Custom - ' Row 0'    (L239, T213, R264, B234)
   |    | ['Custom4', ' Row 02', ' Row 0Custom2']
   |    | child_window(title=" Row 0", control_type="Custom")
   |    | 
   |    | Custom - 'Data Row 0'    (L264, T213, R352, B234)
   |    | ['Custom5', 'Data Row 0Custom', 'Data Row 0']
   |    | child_window(title="Data Row 0", control_type="Custom")
   |    | 
   |    | Custom - 'Technikas Row 0'    (L352, T213, R484, B234)
   |    | ['Custom6', 'Technikas Row 0', 'Technikas Row 0Custom']
   |    | child_window(title="Technikas Row 0", control_type="Custom")
   |    | 
   |    | Custom - 'Paskyros numeris Row 0'    (L484, T213, R638, B234)
   |    | ['Custom7', 'Paskyros numeris Row 0Custom', 'Paskyros numeris Row 0']
   |    | child_window(title="Paskyros numeris Row 0", control_type="Custom")
   |    | 
   |    | Custom - 'Objektas-Adresas Row 0'    (L638, T213, R762, B234)
   |    | ['Custom8', 'Objektas-Adresas Row 0', 'Objektas-Adresas Row 0Custom']
   |    | child_window(title="Objektas-Adresas Row 0", control_type="Custom")
   |    | 
   |    | Custom - 'Klientas Row 0'    (L762, T213, R866, B234)
   |    | ['Custom9', 'Klientas Row 0Custom', 'Klientas Row 0']
   |    | child_window(title="Klientas Row 0", control_type="Custom")
   |    | 
   |    | Custom - 'Stadija Row 0'    (L866, T213, R997, B234)
   |    | ['Custom10', 'Stadija Row 0', 'Stadija Row 0Custom']
   |    | child_window(title="Stadija Row 0", control_type="Custom")
   |    | 
   |    | Custom - 'Darbu vykdytojas Row 0'    (L997, T213, R1147, B234)
   |    | ['Custom11', 'Darbu vykdytojas Row 0', 'Darbu vykdytojas Row 0Custom']
   |    | child_window(title="Darbu vykdytojas Row 0", control_type="Custom")
   |    | 
   |    | Custom - 'Projektas Row 0'    (L1147, T213, R1348, B234)
   |    | ['Custom12', 'Projektas Row 0', 'Projektas Row 0Custom']
   |    | child_window(title="Projektas Row 0", control_type="Custom")
   | 
   | Custom - 'Row 1'    (L199, T234, R1348, B255)
   | ['Custom13', 'Row 1', 'Row 1Custom']
   | child_window(title="Row 1", control_type="Custom")
   |    | 
   |    | Custom - ' Row 1'    (L199, T234, R239, B255)
   |    | ['Custom14', ' Row 1Custom', ' Row 1', ' Row 1Custom0', ' Row 1Custom1', ' Row 10', ' Row 11', ' Row 100', ' Row 101', ' Row 110', ' Row 111']
   |    | child_window(title=" Row 1", control_type="Custom")
   |    | 
   |    | Custom - ' Row 1'    (L239, T234, R264, B255)
   |    | ['Custom15', ' Row 1Custom2', ' Row 12', ' Row 120', ' Row 121']
   |    | child_window(title=" Row 1", control_type="Custom")
   |    | 
   |    | Custom - 'Data Row 1'    (L264, T234, R352, B255)
   |    | ['Custom16', 'Data Row 1Custom', 'Data Row 1']
   |    | child_window(title="Data Row 1", control_type="Custom")
   |    | 
   |    | Custom - 'Technikas Row 1'    (L352, T234, R484, B255)
   |    | ['Custom17', 'Technikas Row 1Custom', 'Technikas Row 1']
   |    | child_window(title="Technikas Row 1", control_type="Custom")
   |    | 
   |    | Custom - 'Paskyros numeris Row 1'    (L484, T234, R638, B255)
   |    | ['Custom18', 'Paskyros numeris Row 1', 'Paskyros numeris Row 1Custom']
   |    | child_window(title="Paskyros numeris Row 1", control_type="Custom")

Thanks

1 Answers1

0

You may not like the answer, but current situation is not good for your task.

  1. If you have no access to the app source code, it's impossible to get texts using pywinauto.

  2. But if you have, it requires to implement so called Server Provider for this control type (described in the UI Automation API docs on MSDN). Though I never tried this way.

  3. Another theoretically possible approach should use managed DLL injection to access internal properties of custom .NET controls. I know TestComplete, Squish and UFT use this way. But they are all very expensive. Never heard about any open source tool using this approach.

P.S. Screen text recognition is also theoretically possible way, but the tools are not mature enough to recognize it reliably.

Vasily Ryabov
  • 9,386
  • 6
  • 25
  • 78
  • Today I tested using UIpath rpa tool and it recognizes text in a cell. Is it because that tool is built using c#? – Dainius Miežiūnas Oct 02 '17 at 13:50
  • This is also commercial tool. Most probably it uses DLL injection to get internal properties of .NET controls. C# doesn't matter so much. We're doing intial research for DLL injection approach. It should be possible in Python also. But it's hard to assign deadlines, because pywinauto is a hobby project for all members of our team. – Vasily Ryabov Oct 02 '17 at 17:43
  • Okay, now I get it, thanks. I really like python and pywinauto, I hope it will grow huge :) – Dainius Miežiūnas Oct 03 '17 at 06:48
  • P.S When I print controls on my personal windows 10 laptop the cell control types are DataItem – Dainius Miežiūnas Oct 03 '17 at 06:49
  • Hmm... then it may work on Win10 using standard methods. :) – Vasily Ryabov Oct 03 '17 at 09:14
  • I'll try it at home, I hope I'll find a way on how to achieve it :) – Dainius Miežiūnas Oct 04 '17 at 10:28
  • @VasilyRyabov After 4 years, is this already true? I need to interact with a grid that appears as a 'Pane' without any child objects. My workaround is to navigate through it using keyboard arrows, CTRL+C and reading data from the clipboard. It is a terrible and inefficient solution, but works. Do you have any hint to me? – Diego Queiroz Mar 24 '21 at 12:54
  • Unfortunately I don't think something is changed in GUI apps. – Vasily Ryabov Mar 25 '21 at 18:28