My android app doesn't use gpu with nnapi running onnx model

Question

I'm trying to run my onnx model with gpu by using nnapi in android environment. I'm trying with this code.

/*part of MainActivity.kt*/
/*=====================================================================*/

val modelID = R.raw.mobilenetv2_int8
val mobelbyteArray = resources.openRawResource(modelID).readBytes()

val ortEnv = OrtEnvironment.getEnvironment()

val ortoption = SessionOptions()

ortoption.addNnapi()

var orisession = ortEnv?.createSession(mobelbyteArray,ortoption);

val height:Long = 224
val width:Long = 224
val batch_size = 1

val shape = longArrayOf(1, 3, height, width)
val env = OrtEnvironment.getEnvironment()

val imgData = FloatBuffer.allocate(height.toInt() * width.toInt() * batch_size * 3)

val inputName = orisession?.inputNames?.iterator()?.next()

env.use {
    val tensor = OnnxTensor.createTensor(env, imgData, shape)
    val startTime = SystemClock.uptimeMillis()
    tensor.use {
        val output = orisession?.run(Collections.singletonMap(inputName, tensor))

    }
}

/*=====================================================================*/

/*build.gradle*/
/*=====================================================================*/
plugins {
    id 'com.android.application'
    id 'org.jetbrains.kotlin.android'
}

android {
    namespace 'com.example.onnxrunkt'
    compileSdk 33

    defaultConfig {
        applicationId "com.example.onnxrunkt"
        minSdk 24
        targetSdk 33
        versionCode 1
        versionName "1.0"

        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_11
        targetCompatibility JavaVersion.VERSION_11
    }
    kotlinOptions {
        jvmTarget = '1.8'
    }
    /*
    aaptOptions{
        noCompress 'onnx'
    }

     */
}

dependencies {
    implementation 'com.microsoft.onnxruntime:onnxruntime-android:1.14.0'
    implementation 'androidx.core:core-ktx:1.9.0'
    implementation 'androidx.appcompat:appcompat:1.6.1'
    implementation 'com.google.android.material:material:1.8.0'
    implementation 'androidx.constraintlayout:constraintlayout:2.1.4'
    testImplementation 'junit:junit:4.13.2'
    androidTestImplementation 'androidx.test.ext:junit:1.1.5'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.5.1'
}
/*=====================================================================*/

But this code doesn't use gpu at all. My phone model is "Galaxy s10 5g" with Android 12. How can I use gpu with nnapi? All of onnx model layers support gpu.

I tried adding CPU_DISABLES option to "addnnapi" method. but it doesn't worked.

My android app doesn't use gpu with nnapi running onnx model

0 Answers0