Getting Lots of ANR due to AccessibilityNodeInfo getChild

Question

Getting lot of ANR reports,

"main" prio=5 tid=1 Runnable | group="main" sCount=0 dsCount=0 flags=0 obj=0x72e8a568 self=0xe65da000 | sysTid=20592 nice=0 cgrp=default sched=0/0 handle=0xe9f6b4a8 | state=R schedstat=( 35792446568 9751828904 97371 ) utm=2328 stm=1249 core=2 HZ=100 | stack=0xff6e5000-0xff6e7000 stackSize=8MB | held mutexes= "mutator lock"(shared held) native: pc 00000000002c45b7 /system/lib/libart.so (_ZN3art15DumpNativeStackERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEEiP12BacktraceMapPKcPNS_9ArtMethodEPv+130) native: pc 0000000000355a83 /system/lib/libart.so (_ZNK3art6Thread9DumpStackERNSt3__113basic_ostreamIcNS1_11char_traitsIcEEEEbP12BacktraceMapb+202) native: pc 0000000000351f67 /system/lib/libart.so (_ZNK3art6Thread4DumpERNSt3__113basic_ostreamIcNS1_11char_traitsIcEEEEbP12BacktraceMapb+34) native: pc 00000000003698df /system/lib/libart.so (_ZN3art14DumpCheckpoint3RunEPNS_6ThreadE+654) native: pc 000000000035662b /system/lib/libart.so (_ZN3art6Thread21RunCheckpointFunctionEv+298) native: pc 00000000003bbcff /system/lib/libart.so (_ZN3art16JniMethodFastEndEjPNS_6ThreadE+46) native: pc 0000000000403fdb /system/framework/arm/boot-framework.oat (Java_android_os_Parcel_nativeReadInt__J+114) at android.os.Parcel.nativeReadInt (Native method) at android.os.Parcel.readInt (Parcel.java:1966) at android.os.Parcel.readExceptionCode (Parcel.java:1906) at android.os.Parcel.readException (Parcel.java:1885) at android.accessibilityservice.IAccessibilityServiceConnection$Stub$Proxy.findAccessibilityNodeInfoByAccessibilityId (IAccessibilityServiceConnection.java:447) at android.view.accessibility.AccessibilityInteractionClient.findAccessibilityNodeInfoByAccessibilityId (AccessibilityInteractionClient.java:286) at android.view.accessibility.AccessibilityNodeInfo.getChild (AccessibilityNodeInfo.java:959) at MyAccessibilityService.traverseNode (MyAccessibilityService.java:94) at MyAccessibilityService.traverseNode (MyAccessibilityService.java:100) at MyAccessibilityService.traverseNode (MyAccessibilityService.java:100) at MyAccessibilityService.traverseNode (MyAccessibilityService.java:100) at MyAccessibilityService.collectTextNodes (MyAccessibilityService.java:69) at MyAccessibilityService.onAccessibilityEvent (MyAccessibilityService.java:384) at android.accessibilityservice.AccessibilityService$2.onAccessibilityEvent (AccessibilityService.java:1527) at android.accessibilityservice.AccessibilityService$IAccessibilityServiceClientWrapper.executeMessage (AccessibilityService.java:1712) at com.android.internal.os.HandlerCaller$MyHandler.handleMessage (HandlerCaller.java:37)

    "main" prio=5 tid=1 Native
      | group="main" sCount=1 dsCount=0 flags=1 obj=0x72e8a568 self=0xe65da000
      | sysTid=11518 nice=0 cgrp=default sched=0/0 handle=0xe9f6b4a8
      | state=S schedstat=( 34572861602 18459176949 199877 ) utm=2350 stm=1106 core=1 HZ=100
      | stack=0xff6e5000-0xff6e7000 stackSize=8MB
      | held mutexes=
      native: pc 0000000000018dac  /system/lib/libc.so (syscall+28)
      native: pc 00000000000b3729  /system/lib/libart.so (_ZN3art17ConditionVariable16WaitHoldingLocksEPNS_6ThreadE+88)
      native: pc 00000000003bbbe3  /system/lib/libart.so (_ZN3artL12GoToRunnableEPNS_6ThreadE+306)
      native: pc 00000000003bba81  /system/lib/libart.so (_ZN3art12JniMethodEndEjPNS_6ThreadE+8)
      native: pc 00000000007ca401  /system/framework/arm/boot-framework.oat (Java_android_os_BinderProxy_transactNative__ILandroid_os_Parcel_2Landroid_os_Parcel_2I+144)
      at android.os.BinderProxy.transactNative (Native method)
      at android.os.BinderProxy.transact (Binder.java:748)
      at android.accessibilityservice.IAccessibilityServiceConnection$Stub$Proxy.findAccessibilityNodeInfoByAccessibilityId (IAccessibilityServiceConnection.java:446)
      at android.view.accessibility.AccessibilityInteractionClient.findAccessibilityNodeInfoByAccessibilityId (AccessibilityInteractionClient.java:286)
      at android.view.accessibility.AccessibilityNodeInfo.getChild (AccessibilityNodeInfo.java:959)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:94)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:100)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:100)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:100)
      at MyAccessibilityService.collectTextNodes (MyAccessibilityService.java:69)
      at MyAccessibilityService.onAccessibilityEvent (MyAccessibilityService.java:384)
      at android.accessibilityservice.AccessibilityService$2.onAccessibilityEvent (AccessibilityService.java:1527)
      at android.accessibilityservice.AccessibilityService$IAccessibilityServiceClientWrapper.executeMessage (AccessibilityService.java:1712)
      at com.android.internal.os.HandlerCaller$MyHandler.handleMessage (HandlerCaller.java:37)
      at android.os.Handler.dispatchMessage (Handler.java:105)


    "main" prio=5 tid=1 Runnable
      | group="main" sCount=0 dsCount=0 flags=0 obj=0x73bf7568 self=0xe7445000
      | sysTid=18941 nice=0 cgrp=default sched=0/0 handle=0xeae2e4a8
      | state=R schedstat=( 169234820933 190791369297 635928 ) utm=11752 stm=5170 core=0 HZ=100
      | stack=0xff795000-0xff797000 stackSize=8MB
      | held mutexes= "mutator lock"(shared held)
      at java.util.ArrayList.clear (ArrayList.java:565)
      at android.view.accessibility.AccessibilityNodeInfo.init (AccessibilityNodeInfo.java:3211)
      at android.view.accessibility.AccessibilityNodeInfo.obtain (AccessibilityNodeInfo.java:3015)
      at android.view.accessibility.AccessibilityCache.getNode (AccessibilityCache.java:231)
    - locked <0x0f41ae94> (a java.lang.Object)
      at android.view.accessibility.AccessibilityInteractionClient.findAccessibilityNodeInfoByAccessibilityId (AccessibilityInteractionClient.java:272)
      at android.view.accessibility.AccessibilityNodeInfo.getChild (AccessibilityNodeInfo.java:959)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:84)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:85)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:85)
      at MyAccessibilityService.traverseNode (MyAccessibilityService.java:85)
      at MyAccessibilityService.collectTextNodes (MyAccessibilityService.java:66)
      at MyAccessibilityService.onAccessibilityEvent (MyAccessibilityService.java:360)
      at android.accessibilityservice.AccessibilityService$2.onAccessibilityEvent (AccessibilityService.java:1527)
      at android.accessibilityservice.AccessibilityService$IAccessibilityServiceClientWrapper.executeMessage (AccessibilityService.java:1712)
      at com.android.internal.os.HandlerCaller$MyHandler.handleMessage (HandlerCaller.java:37)
      at android.os.Handler.dispatchMessage (Handler.java:105)

"main" prio=5 tid=1 TimedWaiting
  | group="main" sCount=1 dsCount=0 flags=1 obj=0x73228568 self=0xed55a000
  | sysTid=19996 nice=0 cgrp=default sched=0/0 handle=0xed8414a8
  | state=S schedstat=( 39022541552 53623486893 130046 ) utm=3252 stm=648 core=2 HZ=100
  | stack=0xff636000-0xff638000 stackSize=8MB
  | held mutexes=
  at java.lang.Object.wait (Native method)
- waiting on <0x0b44b730> (a java.lang.Object)
  at java.lang.Object.wait (Object.java:422)
  at android.view.accessibility.AccessibilityInteractionClient.waitForResultTimedLocked (AccessibilityInteractionClient.java:687)
  at android.view.accessibility.AccessibilityInteractionClient.getFindAccessibilityNodeInfosResultAndClear (AccessibilityInteractionClient.java:582)
- locked <0x0b44b730> (a java.lang.Object)
  at android.view.accessibility.AccessibilityInteractionClient.findAccessibilityNodeInfoByAccessibilityId (AccessibilityInteractionClient.java:291)
  at android.view.accessibility.AccessibilityInteractionClient.getRootInActiveWindow (AccessibilityInteractionClient.java:160)
  at android.accessibilityservice.AccessibilityService.getRootInActiveWindow (AccessibilityService.java:572)
  at MyAccessibilityService.onAccessibilityEvent (MyAccessibilityService.java:370)

My code

ArrayList<AccessibilityNodeInfo> collectNodes(AccessibilityNodeInfo node) {

    ArrayList<AccessibilityNodeInfo> nodeInfoArrayList = new ArrayList<>();
    try {
        int childCount = node.getChildCount();
        for (int index = 0; index < childCount; index++) {
            AccessibilityNodeInfo childNode = node.getChild(index);


            traverseNode(childNode);
            if (childNodes != null && childNodes.size() > 0) {
                nodeInfoArrayList.addAll(childNodes);
                childNodes.clear();
            }
        }
    }
    catch (Exception e){
        e.printStackTrace();
    }
    return nodeInfoArrayList;
}

private void traverseNode(AccessibilityNodeInfo node) {
    try {
        AccessibilityNodeInfo edittextNode = null;
        if (null == node)
            return;

        final int count = node.getChildCount();
        if (count > 0) {
            for (int i = 0; i < count; i++) {
                AccessibilityNodeInfo childNode = node.getChild(i);
                if (childNode == null) {
                    node.recycle();
                    return;
                }
                else {
                    traverseNode(childNode);
                }
            }
        } else {

        }
    }
    catch (Exception error){
        error.printStackTrace();
    }
}

even more issues in Android 8.0. Any idea how to deal with these ANR's.

config.xml file

<accessibility-service
    xmlns:android="http://schemas.android.com/apk/res/android"

    android:accessibilityEventTypes="typeWindowStateChanged|typeViewFocused|typeWindowContentChanged"

    android:notificationTimeout="100"
    android:canRetrieveWindowContent="true"

    android:accessibilityFeedbackType="feedbackGeneric"
    android:accessibilityFlags="flagReportViewIds|flagRequestEnhancedWebAccessibility"
    android:description="Test"/>

please note almost all the ANR report from Android 8.0 devices.

AccessibilityNodeInfo nodeInfo = getRootInActiveWindow(); if(nodeInfo != null){ ArrayList list = collectNodes(nodeInfo); } — Nikhil, Oct 23 '17 at 06:04
Consider running your app at [StrictMode](https://developer.android.com/reference/android/os/StrictMode.html). It can help detect cases where an ANR error might occur. — RonTLV, Oct 29 '17 at 13:48
Already test app in StrictMode but unable to reproduce the ANR. — Nikhil, Oct 30 '17 at 04:01

Nikola Minoski · Answer 1 · 2017-10-30T19:11:01.077

1

The first important thing is to recycle all the nodes allocated by getChild, findAccessibilityNodeInfoByAccessibilityId or any other function. So you should have something like this:

AccessibilityNodeInfo findFirstViewByText(AccessibilityNodeInfo rootNode, String text) {
        if (rootNode == null) return null;
        if (rootNode.getText() != null && rootNode.getText().toString().equals(text))
            return rootNode;
        int childCount = rootNode.getChildCount();
        for (int i = 0; i < childCount; i++) {
            AccessibilityNodeInfo tmpNode = rootNode.getChild(i);
            if (tmpNode != null) {
                AccessibilityNodeInfo res = findFirstViewByText(tmpNode, text);
                if (res != null) {
                    return res;
                } else {
                    tmpNode.recycle();
                }
            }
        }
        return null;
    }
}

node = findFirstViewByText(rootNode, "something");
if(node != null) {
  node.performAction(AccessibilityNodeInfo.ACTION_CLICK);
  node.recycle();
}

Keep in mind that onAccessibilityEvent may be called too often while you interact with the target application and also your processing in this function is blocking UI Thread on the target application. You can overcome this by:
- Throttling the calls to onAccessibilityEvent i.e. don't execute your code if the last execution happened less than x milliseconds ago, but make sure that last call will trigger execution of your code.
- Improve the performance of your code by avoiding unnecessary traversing. Define states so the app will look only for the nodes relevant for the current state.
Feel free to share more of your code so i/we can understand what exactly is your goal, that way we can be more helpful. As i can see from your log, there are several threads traversing the virtual dom of accessibility service and they are causing deadlock.

edited Oct 30 '17 at 19:11

answered Oct 25 '17 at 14:53

Nikola Minoski

416
5
7

+1 to throttling. I'm still curious to get a look at his service_config xml file. -1 to recyling all nodes. You are correct about accessibilityNodeInfo.find..... but not getRootInActiveWindow. If you don't understand the difference between these calls, please dig into AOSP Source. Basically, one returns a copy of nodes, one returns nodes that belong to something else. – MobA11y Oct 25 '17 at 18:47
Let me put this another way. Multiple successive calls on the same UI state to getRootInActiveWindow() return an object with the same memory address. PLEASE DO NOT recycle this and please stop telling people they should. Downvoting until this particular recomendation is removed, otherwise, this is great info... speculative (not your fault), but great info. – MobA11y Oct 25 '17 at 18:59
1

I did some benchmarks with and without recycle, with recycling the code performs a little bit faster and consumes less memory (small differences), but you are right about getRoootInActiveWindow. I got more improvements when i stored the reference returned by getRootInActiveWindow() and used .refresh() method to update its state (in case it is changed). – Nikola Minoski Oct 30 '17 at 19:14
NikolaMinoski @ChrisCM As per NikolaMinoski suggestion "Throttling the calls to onAccessibilityEvent" instead of this if i change "android:notificationTimeout" in config.xml from 100ms to 300ms or 500ms than by default i will get less event of same type. Please correct if i am wrong. – Nikhil Oct 31 '17 at 05:04
You are correct. But, 100ms (.1 sec) is pretty reasonable throttling. Throttle any more than this and you risk missing screen changed events. – MobA11y Oct 31 '17 at 13:40
I use soft throttling to minimize the frequency of the events, both are set to 200ms and the soft throttling is still discarding a lot of unnecessary event. – Nikola Minoski Oct 31 '17 at 15:52

MobA11y · Answer 2 · 2018-05-26T19:43:21.667

0

I'm highly suspicious of your collect nodes logic. It's very convolluted. It actually took me a good while to confirm that it was doing what I thought it was doing, and I'm still not convinced, though mostly because there are a few class/global variables that you have not shared, keep ing me from being 100% sure of anything. Allow me to describe what I think it's doing:

CollectNodes: Collects all descendants of the given node, and returns them in a list.

So, given that description, let me show you a simpler implementation.

public ArrayList<AccessibilityNodeInfo> collectNodes() {
    ArrayList<AccessibilityNodeInfo> results = new ArrayList<>();

    try {
        collectNodes(getRootInActiveWindow(), results);    
    } catch (Exception e) {
        //Handle exceptions, though, if you don't do stupid things exceptions shouldn't happen
    }

    return results;
}

private void collectNodes(AccessibilityNodeInfo nodeInfo, ArrayList<AccessibilityNodeInfo> nodes) {

    if (nodeInfo == null) return;

    if (someFilteringFunctionYouWantToApplyToLimitNodes(nodeInfo)) {
        nodes.add(nodeInfo);
    }

    for (int i = 0; i < nodeInfo.getChildCount(); i++) {
        collectNodes(nodeInfo.getChildAt(i), nodes);
    }
}

NOW, let's talk about some of the things, actually really only one specific thing that I removed from your implementation, and that is this line:

node.recycle();

I believe this is the actual line that is causing you problems. At no point in your code have you created any of your own AccessibilityNodeInfo objects. All of the objects that you're using belong to the the hierarchy of nodes that has been created for you by the Operating system. These nodes should be viewed as a READ ONLY state of your application at a point in time. You are recycling things and reusing them.

edited May 26 '18 at 19:43

answered Oct 23 '17 at 18:13

MobA11y

18,425
3
49
76

apply this approach but still getting ANR, mostly on Android 8.0 – Nikhil Oct 25 '17 at 04:13
Your missing a lot of important info from this. What events are you attaching to, whats your service config look like, etc. – MobA11y Oct 25 '17 at 13:56
1

You should use node.recycle(), from documentation: `Note: It is a client responsibility to recycle the received info by calling recycle() to avoid creating of multiple instances.` – Nikola Minoski Oct 25 '17 at 14:09
When you are done with them. He's recycling them as he's collecting them to be used elsewhere. Also, that documentation is not coming from "getRootInActiveWindow()" call, that is coming from other APIs, such as "findAccessibilityNodeInfos..." which creates new instances of nodes for you, not relying on the current global active state. – MobA11y Oct 25 '17 at 14:34
Who is `he`? If you think about AccessibilityService then you are wrong, it doesn't use reference-count to automatically recycle them. – Nikola Minoski Oct 25 '17 at 15:05
A: recycling is optional. All recycling is going to do is allow a node to be repopulated with information to avoid creating/using more memory. B: no, but it will still destroy them and not memory leak, because this is java and everything is reference counted and garbage collected. C: he: is anyone who is using getRootInActiveWindow() (as HE/OP is, because you can see this in the stack trace) to access the main node content, because if you visit the documentation on said call, you will note the absence of the line that you quoted, leading people to believe this is a good idea... which it isn't. – MobA11y Oct 25 '17 at 17:22
@Nikola: You should dig into AccessibilityInteractionClient and see the differences in how AccessibilityNodeInfos and AccessibilityServices interact with the UIAutomation/Operating system layer. It will become clear to you why over recycling in an Accessibility Service is bad. There is a pretty significant difference between best practices in the UIAutomation APIs are used vs those intended for use by AccessibilityServices, and unfortunately, those two approaches end up mixed together, because Accessibility stuff in the AOSP doesn't get the time it deserves. – MobA11y Oct 25 '17 at 17:38
@ChrisCM config.xml added in question, please check updated question. – Nikhil Oct 30 '17 at 04:03
Nothing about your configuration stands out as problematic, 100ms is perfectly reasonable throttling. Double check that you're not nuking this configuration in your onServiceConnected callback... "TypeViewCofused" is fundamentally different from your other event types. I like Nikola's point about your use of Runnables and threading... – MobA11y Oct 30 '17 at 19:54
@chrisCM I have to disagree with recycle() calling. Take a look at AccessibilityInteractionClient #findAccessibilityNodeInfoByAccessibilityId() which is consequence of getChild(). You will find, they also recycle() items which they searched, but did not return from this method. AccessibilityNodeInfo has pool where are cached items for faster obtaining. If you don't release, you block the pool, thus instances are not so easy to create. You basically blocking another app, even if you don't need. – l0v3 May 26 '18 at 18:55
It doesn't block, it slows. Huge difference. You can get performance benefits from recycling, but you can also get yourself into trouble. Note: they are also continuously changing the documentation (and code) on this. This answer is well out of date. – MobA11y May 26 '18 at 19:43

Getting Lots of ANR due to AccessibilityNodeInfo getChild

2 Answers2

Linked