0

Instructions at http://docs.aws.amazon.com/es_es/devicefarm/latest/developerguide/test-types-android-appium-python.html does not tell anything about adding appium (appium wheel) into the virtualenv needed to build the test_bundle.zip.

If it's not added "py.test --collect-only tests/" run from the virtualenv will obviously fail and test_bundle.zip built without appium will fail on ADF.

So, first, I want to double check that after we install py.test in the virtualenv - "pip install pytest" we need also install appium client - "pip install Appium-Python-Client".

Then tests will run in ADF, but take amazing amount of time just for a single basic test that runs seconds on a physical device. With ADF I need to wait for about 20 minutes for the test to complete and then it shows 5 "Total minutes" for the test run. Does it look right?

Thanks.

Ken
  • 33
  • 6

1 Answers1

0

I work for the AWS Device Farm team.

Short Answer to main question in subject line: Yes

Explanation:

Python virtualenv usage

The confusion seems to around that the virtual environment is needed "just" for packaging. Our recommendation is to actually make sure that your tests run in the virtualenv rather than use it to just package the tests.

This way you will always have all the needed dependencies in your virtualenv and do not have to track dependencies individually.

From our documentation, "We strongly recommend that you set up Python virtualenv for developing and packaging tests so that unnecessary dependencies are not included in your app package."

I will try to highlight this fact in a better way if this was not clear.

Test execution timings

On Device Farm we setup the device and make sure you get a device that is completely clean. We also run a new instance of Appium server for each test. This can add up time when the tests are being executed especially if the tests are really small which takes few seconds which is more than the setup time. If you average out the timings for such tests it can seem to make a difference although you are not charged for the time when we do the cleanup. The device minutes are accounted only after the app is installed and tests are ready to start.

NikofTime
  • 729
  • 3
  • 4
  • Thanks for reply. Please, provide more clarity on timing: 1. Is there a way to short period of "SCHEDULING" phase (before transitioning to the "RUNNING" phase) that sometimes runs for so long (>30minutes) that I need to stop the test. – Ken Aug 18 '16 at 09:54
  • 2. I believe "Total minutes" are actual minutes that were calculated after the test was done and the account will be billed for. They are times higher than amount of time needed to ran exactly same single test on exactly same physical device attached directly to my computer - something like 4 minutes on a local device and 8 minutes in ADF. What's the reason and how this can be improved? Thanks. – Ken Aug 18 '16 at 09:54
  • Sorry for the delay. The actual billed amount is the Total metered minutes from your run which can found when you hover over the run minutes in summary. We are working on making this even faster in future. However, the simplest reasoning is app closing and starting between every test and an new instance of appium server coming up for every test. – NikofTime Jun 06 '17 at 00:40
  • If you found the answer please accept it as the answer. Others could benefit from it. – NikofTime Jun 06 '17 at 00:40