2

Please tell me how do I click in point coordinates in application window? I trying to UI automate my application on OSX 10.10 using JXA technology. In documentation I found that it's possible using click at event. By I'am beginner of JXA and cant find how make a call. Code snippet which I tried in Script Editor:

var app = Application('my_application_path')
app.window.click.at('{100,100}')

Thank you for help

toohtik
  • 1,892
  • 11
  • 27

3 Answers3

6

You can interact with an application's user interface using the System Events application. Here is a script that clicks at certain coordinates in Safari:

// Activate Safari, so you will be able to click like a user

Application("Safari").activate()

// Access the Safari process of System Events

var SystemEvents = Application("System Events")
var Safari = SystemEvents.processes["Safari"]

// Call the click command, sending an array of coordinates [x, y]

Safari.click({ at: [300, 100] })

If you want to click a specific button (or other element of the user interface), it is more appropriate to click that specific element. For example:

// Click the third button of Safari's first window to minimize it

Safari.windows[0].buttons[2].click()

To learn what user interface elements can be interacted with and how, check out the Processes Suite in System Events' scripting dictionary. To open the dictionary, in Script Editor's menu bar, choose Window > Library, then select System Events in the Library window.

syntaxera
  • 254
  • 3
  • 5
1

See https://github.com/dtinth/JXA-Cookbook/wiki/System-Events#clicking-menu-items For example: 

var fileMenu = proc.menuBars[0].menuBarItems.byName('File');
Kevin Suttle
  • 8,358
  • 3
  • 33
  • 37
0

Below is an example of a portion of a script I wrote that automates creating mailboxes (aka folders) in Mail. I ended up using the UI file menus and click because using make() in the Mail DOM had issues for me. Hope it helps someone.

(() => {}
//this is part of a script that automates creating mailboxes (ie folders) in Apple Mail
//I used the file menu UI because when I tried the Mail library and make() method
//there was strange behavior when trying to interact with the new mailbox.  
//However, when creating the new mailboxes thru the file menu, all seems to work fine

    const Mail = Application('Mail');
    const strId = Mail.accounts.byName('Exchange').id();
    const exchange = Mail.accounts.byId(strId);
    const activeClientFolder = exchange.mailboxes.byName('ActiveClient');

    const SysEvents = Application('System Events');
    const mail = SysEvents.processes.byName('Mail');
    //next two lines insure Mail will be open and in front 
    mail.windows[0].actions.byName('AXRaise').perform();
    mail.frontmost = true;
    const mailboxMenu = mail.menuBars[0].menus.byName('Mailbox');

    //below shows EXAMPLES of using .click(), keystroke(), and keyCode()
    let newFolder = function (parentFolder, newFolderName, addTrailingDelay = true) {
        //next line will select the parent mailbox (aka folder) where the new mailbox will be inserted
        Mail.messageViewers[0].selectedMailboxes = parentFolder;
        mailboxMenu.click();
        delay(.2);
        mailboxMenu.menuItems.byName('New Mailbox…').click(); 
        delay(.2);
        SysEvents.keystroke(newFolderName);
        SysEvents.keyCode(36);
        //delay is needed when creating multiple mailboxes with a loop
        if (addTrailingDelay == true){  
            delay(1);
        }
    }
    //now the payoff
    const count = newActiveClients.length;
        for(let i=0;i<count;i++){
            /* Client Root Mailbox */ 
            newFolder(activeClientFolder, newActiveClients[i], true);
        
            /* Client Email Folders */ 
            newFolder(activeClientFolder.mailboxes.byName(newActiveClients[i]), 'Client', true);
            newFolder(activeClientFolder.mailboxes.byName(newActiveClients[i]).mailboxes.byName('Client'), 'Client_FYI_Sent');
            newFolder(activeClientFolder.mailboxes.byName(newActiveClients[i]).mailboxes.byName('Client'), 'Client_FYI_Inbox');
            newFolder(activeClientFolder.mailboxes.byName(newActiveClients[i]).mailboxes.byName('Client'), 'Client_FYI_Client_To'); 
            newFolder(activeClientFolder.mailboxes.byName(newActiveClients[i]).mailboxes.byName('Client'), 'Client_From', false);   
        }   
})()
Rmattson
  • 1
  • 1