3

I'm having trouble getting my test suite to pass in full when it runs on CircleCI. Everything will pass locally when run in the suite and when run individually. I eventually came across the issue that was causing problems, the Apple Notification/Location permission dialog. The tests still run, but any test that is expecting an alert to show fails because the Apple dialog is still on the screen. This basically happens every time you run the full suite on a new device or delete the app.

My question, what is the best way to deal with these dialogs? I'm pretty sure it is impossible to write the UI tests around it since all UI tests are pretty strict about ordering and what is on screen. The tests are randomized and the dialog only shows for the first test that calls for it, the rest won't need to worry about it.

My current thinking, but unsure how to proceed:

  • Mock the request for location/push notifications so it never triggers the dialog. Difficult for me so far as I have a class that does all the work for push and location. Do I mock against Apple classes or my own?
  • Write a seperate target for tests that only has 1 test that triggers the dialog and can close it. This would only be used when run on the CI server. This may not work as the same simulator may not be used between test runs. I would think it would, but no guarantees.
  • Add debug code to live code to bypass some of these dialogs and permissions. I already have code when run in DEBUG for notifications since you never actually get a successful token on the simulator. I just stub a fake return when run in debug so I can continue testing. What's a a little more? I'm not honestly considering this one if I can absolutely help it. There is already enough "testing" code in the live codebase that I'd like to prevent any more if at all possible.

Would love some feedback or examples on how to proceed if anyone has any ideas.

Setup Details:

  • ObjC
  • Latest version of Xcode, supports iOS 8 and up
  • Using KIF and OCMock for testing
Bill Burgess
  • 14,054
  • 6
  • 49
  • 86

2 Answers2

0

We had a couple ways to workaround this:

  1. In beforeEach/setUp, check the existence of the alert box and then use KIF's API to ack.
  2. https://github.com/plu/JPSimulatorHacks just works!
  3. Use a XCUITest to trigger permission flows and turn all permission on before moving to the target test bundle.
Oliver Hu
  • 148
  • 7
0

This answer is woefully late to my own question, but we did manage to solve this by using wix/AppleSimulatorUtils. Locally this was easy to resolve by running the app and approving any pop-ups for Location or Notifications. They won't come back until you run on a new sim. For command line/CI runs, we would use this as part of the setup. Pre-approve any permissions for the device and they won't show up during the test run. This tool has saved me many countless hours and I recommend it completely.

Bill Burgess
  • 14,054
  • 6
  • 49
  • 86