Setting up a series of tests I decided for Appium Python as I saw that it had support in both AWS Device Farm and Microsoft App Center.
I am making a one device test run on AWS and I see that every unit test takes 1:30 minutes to be done. Since I have 26 tests that would be 40+ minutes per test run per device.
For comparison, I can run the same test locally (i7, 16GB) in around 9 minutes, which I find acceptable.
At first, I thought that AWS was ignoring my desired capabitilies and installing the APK on every test, however in the log I see that the requests to the Appium server are taking forever.
2018-01-26 03:13:29:605 - [HTTP] <-- GET /wd/hub/status 200 40 ms - 83
2018-01-26 03:14:05:721 - [HTTP] <-- POST /wd/hub/session 200 35503 ms - 1418
2018-01-26 03:14:05:845 - [HTTP] <-- POST /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/timeouts/implicit_wait 200 59 ms - 76
2018-01-26 03:14:18:612 - [HTTP] <-- POST /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/appium/app/launch 200 12763 ms - 76
2018-01-26 03:14:22:786 - [HTTP] <-- POST /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/element 200 4162 ms - 87
2018-01-26 03:14:23:352 - [HTTP] <-- POST /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/element/1/click 200 560 ms - 76
2018-01-26 03:14:24:006 - [HTTP] <-- POST /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/element 200 650 ms - 87
2018-01-26 03:14:24:264 - [HTTP] <-- POST /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/element/2/click 200 234 ms - 76
2018-01-26 03:14:25:205 - [HTTP] <-- POST /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/element 200 924 ms - 87
2018-01-26 03:14:25:329 - [HTTP] <-- POST /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/element 200 119 ms - 87
2018-01-26 03:14:25:444 - [HTTP] <-- POST /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/element 200 111 ms - 87
2018-01-26 03:14:35:523 - [HTTP] <-- GET /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/screenshot 200 10062 ms - 5261738
2018-01-26 03:14:36:904 - [HTTP] <-- POST /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/touch/perform 200 796 ms - 76
2018-01-26 03:14:47:256 - [HTTP] <-- GET /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/screenshot 200 10319 ms - 5261770
2018-01-26 03:14:49:199 - [HTTP] <-- POST /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5/appium/app/close 200 1139 ms - 76
2018-01-26 03:14:52:874 - [HTTP] <-- DELETE /wd/hub/session/7243baaa-affd-4230-95cd-6f29a41ff6f5 200 3546 ms - 76
Here you can see:
- 35 seconds to create a session
- 13 seconds to launch
- 10 seconds per screenshot. (20 seconds total)
That is 60 seconds of the 90 seconds in tasks outside the test.
Is this the expected performance? I am using a Samsung S8 if it makes a difference.
