6

I am using a Java application at work. I need to send clicks to buttons and fill in textboxes. I would like these actions to happen in the background. The ahk_class of the window is SunAwtFrame, and none of the controls are exposed to WindowSpy.

I have used JavaFerret to determine that the first button I want to press has an AccessibleAction

Version Information:
    Java virtual machine version: 1.7.0_25
    Access Bridge Java class version: 1.7.0_25
    Access Bridge Java DLL version: AccessBridge 2.0.2
    Access Bridge Windows DLL version: AccessBridge 2.0.2

AccessibleContext information:
    Name:  New Call
    Description:  Place a new call
    Role:  push button
    Role in en_US locale:  push button
    States:  enabled,focusable,visible,showing,opaque
    States in en_US locale:  enabled,focusable,visible,showing,opaque
    Index in parent:  1
    Children count:  0
    Bounding rectangle:  [288, 317, 385, 376]
    Top-level window name:  Phone Assistant: 
    Top-level window role:  frame
    Parent name:  
    Parent role:  panel
    Visible descendents count:  0

AccessibleIcons info:
    Number of icons:  1
    Icon 0 description: jar:http://proxy.m5net.com/vox/pa/receptioncenter.jar!/resources/phone.png
    Icon 0 height: 26
    Icon 0 width: 27

AccessibleActions info:
    Number of actions:  1
    Action 0 name: click

Accessible Value information:
    Current Value:  0
    Maximum Value:  1
    Minimum Value:  0

The accessibility docs tell me that I should be to tell the object to perform that action, but I don't know how to do this.

Preferably with AutoHotkey, how can this be done?

Rizwan
  • 103
  • 4
  • 24
Brett Banks
  • 170
  • 2
  • 8
  • I want to use AutoHotKey with a java application as well. Did you ever find a solution to this? – Dave Mar 04 '15 at 22:35
  • Unfortunately I did not. I ended up using a hacky solution the AHK script would pull the Java window forward, interact with the inputs and buttons using relative coordinates, and then push the Java window back again. – Brett Banks Mar 05 '15 at 00:19
  • If you need to check if a Java app has finished loading, you can try [checking the mouse status](https://stackoverflow.com/a/23708478/3357935) or the [color of a specific pixel on the page](https://stackoverflow.com/a/23947684/3357935). – Stevoisiak Sep 29 '17 at 16:08
  • You may be able to use the Java Access Bridge API. (See [How to send to unseen controls in a Java app](https://autohotkey.com/board/topic/95343-how-to-send-to-unseen-controls-in-a-java-app/?p=601593)) – Stevoisiak Oct 02 '17 at 20:49
  • You could make it fullscreen, then interact with specific coordinates. – Xiddoc Mar 27 '21 at 13:37

1 Answers1

0

One option is to set the CoordMode to Relative (Or Window which does the same thing), and then you can use Click or MouseMove with hard-coded pixels relative to the top left corner of the window. You can find the coordinates you need with the Window Spy.

Another option, if you're unsure the hard-coded pixels will be reliable, is you can use the FindText library which allows you to specify what the text/icon is within the window and automate clicks based off of a certain number of pixels nearby that. So for example, if there's always the text "Name:" and an input box to the right of it, you can build a search for "me: [" (left edge of the input box pixels) and then click to the right of that. The library is impressive and uses the built-in ImageSearch command in AutoHotkey.

The successor of this library is called AutomateMyTask, which I haven't tried yet, but it claims that you can build your whole click script within a gui that this tool provides. Check it out!

Dean Householder
  • 549
  • 1
  • 7
  • 13