All Android IPC by binder blocked

Question

I got a problem where all RPC(whether provider or aidl) call into my service app blocked.

Functions client called are simple enough that I'm sure would not block for hours.(and has no dead lock)

Dive into binder driver, found some info in /sys/kernel/debug/binder.

-from bidner/stats.txt, found no binder thread is "ready":

proc 918
threads: 23
requested threads: 0+15/15
ready threads 0

-from binder/proc/918, found some incoming transaction and pending transactions. -from anr info,found high kernel cpu usage

CPU usage from 6506ms to 225ms ago:
   90% 918/com.evideostb.cbb.cbbship: 4.1% user + 86% kernel / faults: 11165203 minor
CPU usage from 3444ms to 3956ms later:
   100% 918/com.evideostb.cbb.cbbship: 0% user + 100% kernel / faults: 1001251 minor
     100% 977/Thread-35507: 0% user + 100% kernel

-try to attach debuger to android process via android-studio, found my process been invisble to android-studio while i can still ps it.

-try to gdb , and thread apply all info, many thread (or that's all java thead) print stack like this(seem infinite):

#0  0xb6e45968 in syscall () from G:\kdroid\tmp\lib\libc.so
#1  0xb4ff76a6 in art::ConditionVariable::Wait(art::Thread*) ()
    from G:\kdroid\tmp\lib\libart.so
#2  0xb4ff76a6 in art::ConditionVariable::Wait(art::Thread*) ()
    from G:\kdroid\tmp\lib\libart.so
#3  0xb4ff76a6 in art::ConditionVariable::Wait(art::Thread*) ()
    from G:\kdroid\tmp\lib\libart.so
......

No JDWP thead stack found

No Thread-35507 found

This app is too complex to put all code here, just paste reference code.

-Service:

public void onCreate() {
    super.onCreate();
    Notification notification = new Notification();
    notification.flags = notification.flags
            | Notification.FLAG_NO_CLEAR;
    startForeground(
            getClass().hashCode(),
            notification);
}

-OnBind return aidl like this:

interface ICbbServiceBinder{
    IBinder getCbbService(int serviceEnumOrdinal);
}

implement:

private static SparseArray<IBinder> mBinders = new SparseArray<>();
public static IBinder getFunction(CbbClient.ServiceName name){
    return mBinders.get(name.ordinal());
}

-Client side:

private final SparseArray<IInterface> mCache = new SparseArray<>();
public IInterface getService(ServiceName name){
    if (!mIsInit){
        Log.e(TAG, "getService: cbb remote server down");
        return null;
    }
    int nameOrdinal = name.ordinal();

    synchronized (mCache) {
        IInterface iInterface = mCache.get(nameOrdinal);

        if (iInterface == null) {
            IBinder binder;
            try {
                binder = mCbbServer.getCbbService(nameOrdinal);
            } catch (RemoteException e) {
                Log.e(TAG, "getService: can't get service from cbb remote service");
                return null;
            }

            switch (name) {
                case CBB_CORE:
                    iInterface = ICbbCore.Stub.asInterface(binder);
                    break;
                case ......//select binder by name
                default:
                    iInterface = null;
                    break;
            }
            mCache.put(nameOrdinal, iInterface);
        }

        return iInterface;
    }
}

Then client can use IInterface return by getService to do RPC.