22

I'm trying to run parameterized tests using Android Test Orchestrator. But for some reason parameterized tests won't start. I can run all tests properly without Orchestrator but I need it to clear some data between tests.

This is output from Gradle. It can see all 14 tests but only 12 are executed (missing 2 are parameterized):

Starting 14 tests on Nexus_5X_API_27(AVD) - 8.1.0

Tests on Nexus_5X_API_27(AVD) - 8.1.0 failed: Test run failed to complete. Expected 14 tests, received 12

Test run failed to complete. Expected 14 tests, received 12

Gradle:

android {
  
    defaultConfig {       
        testInstrumentationRunner "foo.bar.CustomRunner"
    }
 
    testOptions {
        unitTests.returnDefaultValues = true
        execution 'ANDROID_TEST_ORCHESTRATOR'
    }
}

dependencies {  
    androidTestImplementation "com.android.support.test:runner:$runnerVersion"   
    androidTestUtil "com.android.support.test:orchestrator:$runnerVersion"
}

Test:

@LargeTest
@RunWith(Parameterized::class)
class ParamTest(val param1: String, val param2: String) {

    companion object {
        @JvmStatic
        @Parameterized.Parameters
        fun data(): Collection<Array<Any>> {
            return listOf(
                    arrayOf("param1", "param2"),
                    arrayOf("param21", "param22")
            )
        }
    }

    @Test
    fun shouldDoSthWithParams() {
       //some test
    }

}

EDIT:

As a workaround sealed class can be used:

@LargeTest
@RunWith(AndroidJUnit4::class)
class ParamTest1 : ParamTest("param1", "param2")

@LargeTest
@RunWith(AndroidJUnit4::class)
class ParamTest2 : ParamTest("param21", "param22")

sealed class ParamTest(val param1: String, val param2: String) {
  
    @Test
    fun shouldDoSthWithParams() {
       //some test
    }

}
Aleksander Mielczarek
  • 2,787
  • 1
  • 24
  • 43
  • Have you found any other solution?, sealed classes don't scale, especially with dynamically generated test data. – TWiStErRob Mar 23 '19 at 19:36
  • @TWiStErRob So, annotation processor can be used to generate ui tests. But it also isn't perfect solution due to extra kapt step before run. – Aleksander Mielczarek Mar 24 '19 at 12:58
  • Ah, nice idea. With Incremental KAPT on the way, it could be a viable solution. Although I would prefer if it's "just worked" out of the box... i.e. https://github.com/android/android-test/issues/215 – TWiStErRob Mar 24 '19 at 13:02

2 Answers2

1

The issue was fixed in version 1.3.0-beta02 of the androidx.test:runner library. See the AndroidX Test 1.3.0 Beta02 release notes which calls this issue out explicitly as fixed.

Adil Hussain
  • 30,049
  • 21
  • 112
  • 147
0

Parameterized tests aren't currently supported in Orchestrator: https://developer.android.com/jetpack/androidx/releases/archive/test#relnotes-20170725-known-issues

Pavel Kataykin
  • 1,527
  • 15
  • 14
  • 1
    **Update:** Parameterized tests are supported as of version `1.3.0-beta02` of the `androidx.test:runner` library. See my answer [here](https://stackoverflow.com/a/67858790/1071320) for a link to the release notes. – Adil Hussain Jun 06 '21 at 11:54