How to use XCTestSuite? (Swift 3)

Question

We have a Jenkins job that runs our iOS UI tests, and I need to be able to ignore some tests (or test classes) based on certain runtime information during a test run. According to the XCTestSuite documentation https://developer.apple.com/reference/xctest/xctestsuite you can create a custom test suite programmatically, but I haven't found any examples of how to actually do so. So how exactly does one go about it?

I've toyed with this with the code below as a starting point, but haven't really gotten anywhere. When the IgnoreThisClass is run in Xcode, the test method in it gets run even though the testCaseCount in BaseTestClass' setUp() reports 0 tests in suite. Do I need to override some init() method instead or do something else? Any tips would be greatly appreciated!

import XCTest

class BaseTestClass: XCTestCase {
    var runThis: Bool?

    override func setUp() {
        if runThis != nil {
            let suite = XCTestSuite(name: "Custom suite")

            if runThis! {
                suite.addTest(self)
            }
            print("Testcases in suite \(suite.testCaseCount)")
            // This causes an error that the suite has already started,
            // even though this call is present in Apple's example documentation.
            //suite.run()
        }
        XCUIApplication().launch()
    }

    override func tearDown() {
       super.tearDown()
    }
}

// Run this.
class RunThisClass: BaseTestClass {

    override func setUp() {
        super.runThis = true
        super.setUp()
    }

    override func tearDown() {
        super.tearDown()
    }

    func testThis() {
        print("Test class was run.")
    }
}

// Don't run this.
class IgnoreThisClass: BaseTestClass {

    override func setUp() {
        super.runThis = false
        super.setUp()
    }

    override func tearDown() {
        super.tearDown()
    }

    func testIgnoreThis() {
        print("Ignored class was run.")
    }
}

Can you give an example of when you would want to dynamically rebuild the list of tests that you run? Can you simply move specific tests to separate test bundle and run it conditionally? — pronebird, Jan 11 '17 at 15:02
For example, we found out that some of the tests are failing/behaving differently in devices that are running iOS 10 but work as before in devices that are still running iOS 9, while the app functionality itself hasn't changed. Hence, to cut down the already long test runtimes we'd like to stop the failing tests from running in iOS 10 devices - as they are false positives - until we've fixed them. We already have more than 100+ tests and are writing more each day, so we'd rather not maintain separate test bundles if we can avoid it. — sjalv, Jan 12 '17 at 10:16

score 1 · Accepted Answer · answered Jan 12 '17 at 08:26

1

To achieve what you want, you won't be able to initialise the XCTestSuite from within an XCTestCase subclass. This is because in order for an XCTestCase to run its tests (and therefore setup), it must be part of an XCTestSuite already - so initialising a custom test suite inside the already-initialized test won't work.

Xcode automatically initializes the necessary test classes, including an XCTestSuite, so you don't have control over when it's initialized and aren't able to send it any custom parameters.

Your best bet would be to add an extension for XCTestSuite, and add the evaluation of whether to run test cases included within the suite there.

answered Jan 12 '17 at 08:26

Oletha

7,324
1
26
46

Thanks, so I figured as the custom suite's suite.run() method kept complaining that the suite was already running. The debugger also showed that the default suite was already created. I'll see about extending the XCTestSuite. – sjalv Jan 12 '17 at 10:17
@sjalv Any Luck with that? – Abhishek Bedi Jul 25 '22 at 19:47

How to use XCTestSuite? (Swift 3)

1 Answers1