-1

I tried searching and could not find the maximum time limit for an XCTest. I wanted to write a test that is going to run for 1 - 2 hours. Can I do this using wait(for: [CompletionExpectation], timeout: 3600)? Or will the test fail due to upper limit on test run time?

And if XCtest can't run such long tests then is there a different testing framework I can use? It would be nice if the testing framework belongs to one of these options: Working with iOS Tests in AWS Device Farm

Parth
  • 2,682
  • 1
  • 20
  • 39
  • AFAIK XCTestExpectation can wait for an hour. But is test is going to do anything in these 1-2 hours, or is it going to just idle and wait for completion of that test? if the later, there's really no point to introduce the potential instability and overhead of this wait (first of all such test would hold the CI resources for 1-2 hours for no reason, secondly, it has a good chance to fail often, since any issue in the infrastructure would break it). Instead initialize this background work; and then check the results of that work in scheduled run 2 hours later. – timbre timbre Mar 31 '21 at 18:11
  • And another question you should be asking is: are you testing the right thing. If iOS is not involved in this 1-2 hours of work, why are you testing this from iOS side? how does this connect to your actual app? i.e. do you expect the app hanging there for 1-2hours waiting for response? – timbre timbre Mar 31 '21 at 18:14
  • According to [executionTimeAllowance](https://developer.apple.com/documentation/xctest/xctestcase/3526064-executiontimeallowance) the default value is 10 mins. Are you saying it can be increased to any time I want? And yeah that is a good question, and my testing is limited to what Apple offers. – Parth Mar 31 '21 at 19:17
  • yes, you can change that setting. But the longer it is the less stable your test is. So before fighting apple, I would really make sure it's needed. iOS tests should represent something that can happen realistically on iOS. iOS app not doing anything for 1 hour, and still alive - doesn't sound like a realistic scenario – timbre timbre Mar 31 '21 at 22:09

1 Answers1

1

I am an engineer on the AWS Device Farm team. You can see the time limits for test execution on Device Farm here: https://docs.aws.amazon.com/devicefarm/latest/developerguide/limits.html. XCTest frameworks can be extended for several hours through looping or with the use of https://developer.apple.com/documentation/xctest/xctestcase/3526064-executiontimeallowance as mentioned before.

For better assistance on AWS Device Farm specific issues, I suggest utilizing the AWS Device Farm forums as a medium to contact the service team: https://forums.aws.amazon.com/forum.jspa?forumID=193

Andy

bad-hambo
  • 49
  • 2