0

I have an Xcode 5 CI server running my XCTest unit tests.

My test cases rely on breakpoints to trigger specific actions. These actions are essential to the running of the tests.

Everything passes if I run the tests locally. The problem is: when a bot runs the tests on the server it seems as though breakpoints are ignored.

I tried a sample breakpoint, with an alert sound, just for testing. I shared the breakpoint and committed the shared breakpoint to the project's git repository. The bot correctly checks out the project with the breakpoint included (I can verify this by examining the project in /Library/Server/Xcode/Data/BotRuns/Cache/...).

However, when the bot runs the breakpoint is NOT triggered. I don't hear the sound and execution does not pause.

This behaviour obviously makes sense for most cases, but in my specific case - is there any way to configure the bot so that breakpoints are not ignored?

jackslash
  • 8,550
  • 45
  • 56
chris838
  • 5,148
  • 6
  • 23
  • 27

1 Answers1

1

Whether you can enable this or not having your tests rely on something external to the system under test, like a breakpoint, to ensure that your tests pass seems like a broken design to me.

Ideally your tests should be able to run on any machine either in an interactive or non interactive way. As you cant guarantee that Breakpoints have the "Automatically continue after evaluating" flag set then they would seem to definitely not be suited to a non interactive run.

Using breakpoints for testing also then adds a dependency on Xcode for running the tests as other build systems like xcodebuild and xctool might not even understand breakpoints in the project file.

I would refactor your tests to remove this dependency on breakpoints. If you need help with that it sounds like a great stack overflow question ;)

jackslash
  • 8,550
  • 45
  • 56
  • In this case the system under test includes special hardware connected to our CI server. We need a way of talking to the special hardware from the tests running on the iPhone - I thought debugger breakpoint actions might be a quick and dirty solution! But yes, you are totally right that having this dependency on breakpoints is not a good idea. I've since refactored the tests to use a WiFi interface instead. I'd thought I'd leave this question up because I'm still curious as to whether there is an answer. – chris838 Mar 17 '14 at 08:44