Accessibility Service Cannot Set Text For EditText In Browser Window

Question

I am using an accessibility service to monitor the window for EditText fields. I then need to set the value of these EditText fields. See the following code:

Meta data:

<?xml version="1.0" encoding="utf-8" ?>
<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
    android:accessibilityEventTypes="typeAllMask"
    android:accessibilityFeedbackType="feedbackSpoken"
    android:accessibilityFlags="flagDefault"
    android:notificationTimeout="100"
    android:canRetrieveWindowContent="true"
    android:canRequestEnhancedWebAccessibility="true"/>

In onAccessibilityEvent of the service:

AccessibilityNodeInfo source = event.getSource();
if (source != null & event.getClassName().equals("android.widget.EditText")) {
    Bundle arguments = new Bundle();
    arguments.putCharSequence(AccessibilityNodeInfo
            .ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "some value");
    source.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, arguments);
}

This works properly when viewing another app.

When viewing a website in the android browser I see see that it properly finds the EditText (gets all the way to my performAction code), however, calling the SET_TEXT action on performAction does nothing.

How do I set the text value of a website input field (accessibility service finds it as an EditText)? Do I need to use EnhancedWebAccessibility in some way to inject Javascript? If so, how do I do that? I cannot find any documentation on using EnhancedWebAccessibility.

score 3 · Answer 1 · answered Sep 21 '19 at 00:48

This might be a bit late but might help someone else. the below code checks if lollipop and above and uses the set text method else it uses the clipboard manager. For the clip board manager it takes the value already in the clipboard swaps it for the value you want and then restores the original value.

AccessibilityNodeInfo.ACTION_SET_TEXT

was introduced in android lollipop (21)

if (Build.VERSION_CODES.JELLY_BEAN_MR2 <= Build.VERSION.SDK_INT) {
    nodeInput.refresh();
}

String response = "sometext";
if (Build.VERSION_CODES.LOLLIPOP <= Build.VERSION.SDK_INT) {
    Bundle bundle = new Bundle();
    bundle.putCharSequence(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, response);

    //set the text
    nodeInput.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, bundle);
} else {

    ClipboardManager clipboardManager = (ClipboardManager) activity.getSystemService(Context.CLIPBOARD_SERVICE);
    if (clipboardManager != null) {

        String lastClip = "";
        ClipData clipData = clipboardManager.getPrimaryClip();

        if (clipData != null) {
            lastClip = clipData.getItemAt(0).coerceToText(activity).toString();
        }

        clipboardManager.setPrimaryClip(ClipData.newPlainText("label", response), response));

        if (Build.VERSION_CODES.JELLY_BEAN_MR2 <= Build.VERSION.SDK_INT) {
            nodeInput.performAction(AccessibilityNodeInfo.ACTION_PASTE);
        } else {
            nodeInput.performAction(AccessibilityNodeInfoCompat.ACTION_PASTE);
        }

        clipboardManager.setPrimaryClip(ClipData.newPlainText(lastClip, lastClip));
    }
}

Accessibility Service Cannot Set Text For EditText In Browser Window

1 Answers1