4

it's possible to click (with AccessibilityService) specific buttons that are placed inside Webview? E.g some application A has a webview inside MainActivity. After activity started web content is loaded in webview. Loaded web page contains button with text "Chat". Can I click this button with Accessibility Service? I know I can perform click on Android native widgets (android.widget.Button, TextView etc) but I don't know how to perform click on the button that displayed on web page inside web view. Please help

Rohit Sharma
  • 167
  • 2
  • 14
  • You can do this by overriding the URL which you get after clicking on that button and in that method you do whatever you want to do – Akash Dubey Apr 24 '17 at 10:08
  • My app contain a cursor that's run with its own service when I click on any icon it perform click, if i am out of my application and operating other app that coantain webview then performAction not working . – Rohit Sharma Apr 25 '17 at 10:31
  • @AkashDubey i think Rohit wants to click WITHOUT user touching the screen. Also from a backend service. How will he get the FOCUS on a link using screen coordinates, without touching the screen? – M. Usman Khan Apr 27 '17 at 17:24

1 Answers1

1

This is a complicated answer. Yes, you can click on the views, HOWEVER, it depends on how the web developer implemented the button. If they did something silly like implemented touch down touch up listeners, instead of onClick listeners, it may well be broken. Suggestion: use TalkBack. If it doesn't work with TalkBack, you can't do it. If it does, the following code should work for you:

AccessibilityNodeInfo info = ...;

info.performAction(AccessibilityNodeInfo.ACTION_CLICK);

SideNote: There was rumor of Android implementing the ability, from accessibility services, to fake real hardware events. Thus far, I've only seen this utilized in demos from Android, and only for System apps (with elevated privileges). Keep an eye out for the APIs you need to perfect this, they may in fact be hiding somewhere, but I haven't seen them publicized yet.

MobA11y
  • 18,425
  • 3
  • 49
  • 76
  • In Talkback, if we enable "Single-Tap clicks" then single tap does not work for web pages (after a link is focused by a preceding tap). This shows that performAction(CLICK) doesn't work for web pages. Links work for Double-Tap, but I guess double-tap is internally handled and we dont have the access. – M. Usman Khan Apr 27 '17 at 17:28
  • Single-Tap is experimental, I wouldn't use it as a metric for anything. – MobA11y Apr 27 '17 at 17:29
  • yea but on Single-Tap it does performAction(CLICK) and it works for all other views except for Web pages and Keyboards. It at least shows that performAction(CLICK) wont work on Web pages. ? – M. Usman Khan Apr 27 '17 at 17:33
  • 1
    Yeah, this might be a reasonable test, but the blanket statement that "performAction(CLICK)" does not work on web is false, I just did so on my phone with TalkBack on Google.com. Started a search. As I said in the answer, it depends on how the website implements it's event handlers. Unfortunately in the current accessibility APIs there is no way to reliably fire all potential events that web developers can use to handle clicks from within an AccessibilityService – MobA11y Apr 27 '17 at 17:44
  • Right. Is this limitation documented somewhere? I want to convince my client. Also, We can't simulate the Double-tap of Talkback programmatically (without user require to touch the screen), right? Is this documented? – M. Usman Khan Apr 27 '17 at 17:59
  • No there is no documentation regarding this – Rohit Sharma Apr 27 '17 at 19:02
  • I Always found use .performAction(AccessibilityNodeInfo.ACTION_CLICK); but not getting clearly answer with WebView. – Rohit Sharma Apr 27 '17 at 19:04