-1

I have the below lines of code for which I need to test the performance criteria. But unfortunately Apple Document does not help to get a clear understanding on this. 

func addStrings() {
    let maxCount = 100000
    var array:Array<String> = Array()
    for _ in 0..<maxCount {
        array.append("Test")
    }
}

The performance Test Case Code is listed below.

func testPerformanceExample() throws {
    // This is an example of a performance test case.
    let strings = StringsCreator()
    self.measure {
        // Put the code you want to measure the time of here.
        strings.addStrings()
    }
}

From the result it looks like the first value is 21.3% increase of the average (0.0611s). But what do the rest of the attributes mean? And how do I set (baseline and Max STD Dev) them to an acceptable criteria?

How does it calculate Result is 88% better and what is +- 8%?

enter image description here

TylerP
  • 9,600
  • 4
  • 39
  • 43
NNikN
  • 3,720
  • 6
  • 44
  • 86

1 Answers1

0

To establish the baseline, run the test. You "pass" (because you are given a free pass the first time). The timing information is consolidated. The measurements don't mean anything at that point.

From then on, however, every subsequent time you run the test, you fail if deviation or slowdown from the baseline is too great. If your subsequent test is better and you want to make it a new baseline, the dialog you are displaying in your screenshot has a Set Baseline button that you can use for that purpose.

matt
  • 515,959
  • 87
  • 875
  • 1,141
  • But on what basis the MAX STDDev value should be set? – NNikN Apr 26 '20 at 21:13
  • Just like Base Line, we can also set the value of Max STDDev. – NNikN Apr 26 '20 at 21:34
  • Oh, I see what you mean. Well, you just use whatever you like. As I said in my answer, this is a value where, if your standard deviation _exceeds_ it, it will count as a fail. Each test run is actually multiple runs (ten, I think). So if those are mutually inconsistent beyond the max std dev, you fail. If you have no reason to change it, leave it alone. Perhaps you might start failing and say, well, that wasn't really such a bad run, let's increase the max std dev and let it pass instead. – matt Apr 26 '20 at 21:42