0

I have multiple realizations of the same function contract. Some are naive and straightforward, some are more complex and optimized. I'd like to run them over randomly picked points from input domain using PropSpec.

The question is how to run all redundant realizations and compare output pair-wise. The test should be marked as failed in case computed values differ from one realization to another. If there are more than two realizations it should be possible to decide which one failed based on voting, like in the TMR system

ayvango
  • 5,867
  • 3
  • 34
  • 73

2 Answers2

0

Your test method has to choreograph calling all the individual implementations and then take the vote. That is the only way to ensure that every implementation is tested with the same input and ensure that the output is compared with all of the others.

Bob Dalgleish
  • 8,167
  • 4
  • 32
  • 42
0

Make a generator for your function implementations, and let ScalaCheck randomize both the implementation and the input. Something like this concept code:

type Input = ...
type Output = ...

trait Algorithm {
  def apply(input: Input): Output
}

val funA: Algorithm = ...
val funB: Algorithm = ...
val funC: Algorithm = ...
val funD: Algorithm = ...

import org.scalacheck._
import Prop.BooleanOperators // for the ==> operator

genInput: Gen[Input] = ...

genAlgorithm: Gen[Algorithm] = Gen.oneOf(funA, funB, funC, funD)

propAlgorithm = Prop.forAll(genAlgorithm, genAlgorithm, genInput) {
  case (f, g, x) => (f != g) ==> f(x) == g(x)
}

To make the error reports helpful, you should have also have a sensible toString method on Algorithm.

Rickard Nilsson
  • 1,393
  • 11
  • 11
  • there is a little inconvenience in the suggested method. It have no separate reports for every realization. When test fails I know only that one of them goes out of sync with another and I'm not presented with knowledge which one. And there are bigger issues with this approach: if a test case fail with exception or timed out with endless loop all test cases are aborted. – ayvango Jul 18 '14 at 12:22