Click Button in Toolbar of Other Program

Question

I'm trying to automate some stuff on a legacy application that I don't have the source to. So I'm essentially trying to use the Windows API to click the buttons I'll need on it.

There is a toolbar of type msvb_lib_toolbar that looks like this:

I can get a handle to it (I think) by using this code:

IntPtr window = FindWindow("ThunderRT6FormDC", "redacted");
IntPtr bar = FindWindowEx(window, IntPtr.Zero,"msvb_lib_toolbar",null);

Looking at the docs, it seems I should be able to use SendMessage and the TB_PRESSBUTTON message to click these buttons:

[DllImport("user32.dll")]
public static extern int SendMessage(int hWnd, uint Msg, int wParam, int lParam);

However, I'm not sure how to go about setting the wParam and lParam to click the wanted button on the bar. The documentation doesn't seem to be helping much either.

Could you please advise?

Based on comments, I've also tried UIAutomation. I can locate the toolbar using the following code:

AutomationElement mainWindow = AutomationElement.RootElement.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.NameProperty, "Migration Expert"));
AutomationElement toolbar = mainWindow.FindFirst(TreeScope.Subtree, new PropertyCondition(AutomationElement.ClassNameProperty, "msvb_lib_toolbar"));

But from here, I'm not sure what to do as Spy++ shows no further children of this object:

Loking at the Current property of this AutomationElement I can't seen anything jumping out at me but the BoundingRectangle does seem to indicate that I've found the right element.

Using inspector.exe also doesn't indicate any children on the toolbar.

https://stackoverflow.com/questions/43083781/automation-support-for-visual-basic-6-listview — David Heffernan, Jun 29 '20 at 12:36
I don't know if the length or width really matters. I windows will come up for a short period of time and close and it doesn't matter the size. See : http://www.pinvoke.net/default.aspx/user32/SendMessae%20(User32)MouseClick.html — jdweng, Jun 29 '20 at 12:38
Here's both your problem and solution: [You can't simulate keyboard input with PostMessage](https://devblogs.microsoft.com/oldnewthing/20050530-11/?p=35513). — IInspectable, Jun 29 '20 at 13:08
Use UIAutomation. You can enumerate the children of the toolbar and then use the InvokePattern to invoke the one you like. — Raymond Chen, Jun 29 '20 at 14:54
@RaymondChen doesn't that require the target application to be built to support automation? — ScottishTapWater, Jun 29 '20 at 18:59
If the control is a comctl32 toolbar (which you are already assuming by virtue of the fact that you're sending `TB_PRESSBUTTON`), then it will support automation. Nearly all commercial programs will support automation because that's a requirement for selling software to the US government. — Raymond Chen, Jun 29 '20 at 19:04
@RaymondChen thanks, is `msvb_lib_toolbar` derived from `comctl32` then? — ScottishTapWater, Jun 29 '20 at 19:10
@RaymondChen I honestly don't know, I'm not very familiar with this sort of stuff... I've given UIAutomation a try as you suggested and I can find the `msvb_lib_toolbar` but from there I can't seem to work out how to click the button. Spy++ doesn't indicate any further child elements so I'm a bit stumped at this point. — ScottishTapWater, Jul 01 '20 at 08:53
You want to use [the accessibility inspector](https://stackoverflow.com/questions/40496048/whats-the-difference-of-uispy-exe-and-inspect-exe-from-microsoft-windows-sdk), not Spy. Spy sees only window handles. — Raymond Chen, Jul 01 '20 at 14:24
@RaymondChen thanks, I've just tried that and it's still showing as there being no child elements — ScottishTapWater, Jul 01 '20 at 19:01
Then maybe that application's toolbar isn't accessible. In which case you'll have to do something ad hoc like screen scraping, doing OCR, and then simulating button clicks. — Raymond Chen, Jul 01 '20 at 19:26
@Persistence Use Accessibility tool like Inspect and hover the cursor over the button to see if Inspect can find this button control. If yes, try use [Invoke.Invoke](https://learn.microsoft.com/en-us/windows/win32/winauto/inspect-objects#interacting-with-ui-elements) menu to click the button. — Rita Han, Jul 02 '20 at 09:07
@RitaHan-MSFT it can't find it, the whole toolbar just shows up as a "pane" — ScottishTapWater, Jul 02 '20 at 09:09
@Persistence I mean the button like "new" or "open" instead of the toolbar. If you have not tried that. — Rita Han, Jul 02 '20 at 09:14
@RitaHan-MSFT yeah, it sees the whole toolbar as one thing, no option to get a button — ScottishTapWater, Jul 02 '20 at 09:15
@Persistence It seems not accessible. So simulating mouse input is the way to go. — Rita Han, Jul 02 '20 at 09:23

score 8 · Accepted Answer · answered Jul 01 '20 at 23:03

Not really an ideal solution but I got something quick and dirty working using a combination of pywinauto and pyautogui.

import pyautogui
import subprocess
import sys
import time
import os
from os import path
from glob import glob
from subprocess import check_output


from pywinauto import application


def click_at_image(image):
    location = pyautogui.locateOnScreen(image)
    buttonx, buttony = pyautogui.center(location)
    pyautogui.click(buttonx, buttony)

def get_dcf_filepaths():
    files = []
    start_dir = redacted
    pattern = "*.DCF"
    for dir, _, _ in os.walk(start_dir):
        files.extend(glob(os.path.join(dir, pattern)))
    return files

def get_csv_paths(paths):
    csv_paths = []
    for p in paths:
        csv_paths.append(p.replace(redacted,redacted).replace("DCF","csv").replace("dcf","csv"))
    return  csv_paths


def main():
    app = application.Application().start(redacted)
    files = get_dcf_filepaths()
    csv_paths = get_csv_paths(files)
    time.sleep(3)
    click_at_image("new_button.png") #Open new project
    for i in range(0, len(files)):
        if (path.exists(csv_paths[i])):
            #os.remove(csv_paths[i])
            continue
        time.sleep(1)
        # Click on nxt icon in dialog
        click_at_image("nxt_button.png")
        # Enter file path into OFD
        app.Open.Edit.SetText(files[i])
        pyautogui.press('enter')
        pyautogui.press('enter')
        time.sleep(1)
        # Click on m2c icon in toolbar
        click_at_image("m2c_button.png")
        # Wait for Excel to open
        time.sleep(6)
        # Open Save as dialog and browse
        pyautogui.press('alt')
        pyautogui.press('f')
        pyautogui.press('a')
        pyautogui.press('o')
        time.sleep(2)
        pyautogui.press('backspace')
        # Enter file path
        pyautogui.write(csv_paths[i], interval=0.01)
        #click_at_image("dummy.png")
        # Change file type to CSV and ignore any popups
        click_at_image("dd.png")
        time.sleep(1)
        
        click_at_image("csv.png")
        pyautogui.press('enter')
        pyautogui.press('enter')
        pyautogui.press('enter')
        time.sleep(2)
        # Kill excel
        pyautogui.hotkey('alt', 'f4')
        # Pull main window back to top
        app.top_window().set_focus()
        time.sleep(1)
        # New project
        click_at_image("new_button.png")
        time.sleep(0.50)
        # Don't save last one
        click_at_image("no.png")

if __name__ == "__main__":
    main()

Essentially I had to resort to using screenscraping to click the non-accessible buttons. If this was for something that needed to be more robust, I'd have done this in C# using the Win32 API directly for everything except the screen scraping with some additional checks to wait for windows to appear rather than using dumb timers.

That being said, this works and may be helpful for future readers.

Click Button in Toolbar of Other Program

1 Answers1