2

Can I upload Android Orchestrator .apk to the AWS Device Farm as a 3rd apk (together with main app apk and test-suite apk)? Or application state is not persistent in the Device Farm, so that it doesn't require external Orchestrator at all?

Currently we use local runner for Android Espresso tests. Some tests require app state/data to be cleared between tests, so that we use Orchestrator. According to a console, it is uploaded as a separate apk to the device. AWS Device Farm allows uploading only 2 apks: app itself and a test-suite.

So, how could I use Orchestrator in the AWS Device Farm for tests to have cleared app state/data after each test?

Shurov
  • 400
  • 1
  • 9
  • 20

2 Answers2

0

You could send Orchestrator as an auxiliary apps when using AWS Device Farm.

If you're using the console, this is done during the Specify device state step under Install other apps

If you're using the CLI use auxiliaryApps flag https://docs.aws.amazon.com/cli/latest/reference/devicefarm/schedule-run.html

Scott Theriault
  • 98
  • 8
  • Can someone confirm if it's working? I don't think that it's as simple as uploading a file since when starting tests with Test Orchstrator a different runner is used. When launching in AS you can see that in console logs: `$ adb shell CLASSPATH=$(pm path androidx.test.services) app_process / androidx.test.services.shellexecutor.ShellMain am instrument -r -w -e targetInstrumentation com.myapp.test/myapp.test.MyAndroidJUnitRunner -e clearPackageData true --no-window-animation -e debug false androidx.test.orchestrator/androidx.test.orchestrator.AndroidTestOrchestrator` – Piotr Zawadzki Nov 05 '20 at 10:58
0

Short answer: Uploading the orchestrator seems to work, but you dont need it to get the tests run in separate processes.

Long answer: As Scott's answer said, you can upload and install the orchestrator to the device. You should upload android test services apk as well (test-services-1.4.0-alpha05.apk for example)

Also the Piotr's comment is correct. If you want to use the orchestrator, you should specify your own start command for tests like this: "adb shell "CLASSPATH=$(pm path androidx.test.services) exec app_process / androidx.test.services.shellexecutor.ShellMain am instrument ...". This can be done by using the custom evironment option from the device farm's start test wizard. More info can be found here: https://developer.android.com/training/testing/junit-runner#using-android-test-orchestrator

If you want to get the tests run separatedly, just use the default environment option, as it seems to isolate the tests by default in the aws farm. There's no need to use the orchestrator. Orchestrator is needed only, if you want to use your own custom test runners, or use custom parameters that needs to be passed to the tests.

j4kk1
  • 61
  • 1
  • 3