1

I’m trying out Spec Explorer, and now I have this bug that my test suite is incomplete. I don't get an error or anything, it’s just that I would expect 16 test cases and I only have 11 of them.

The problem I have is with the sample project that is in Spec Explorer 2010. Because I’m new at this I was trying different stuff out with the sample project, so stuff like expanding the range and expanding the double add to quadruple add. This last one was where I noticed that I was missing some test cases. I changed it back to triple add, to watch if the problem was there to. And as I expected I missed a test case again. Only with the triple I expected 8 test cases and I only got 7. The only thing I changed in the code:

machine DoubleAddScenario() : Main where ForExploration = true
{
    (Add(_); Add; Add; ReadAndReset)*
}

I’ve also tried to do this

(Add(_); Add(_); Add(_); ReadAndReset)*

But same problem there. The test case I’m missing is the Add(1); Add(2); Add(1). I’ve also tried calling only this one, and that works, so why am I missing it in my test suite?

Am I doing something wrong, or does Spec Explorer filter something for me? And if it is Spec Explorer where does it make this decision?

Kyra
  • 173
  • 2
  • 16

1 Answers1

1

good question. The reason why the test case is missing is, that Spec Explorer uses step (transition) coverage and not full path coverage as coverage criterion. So you will find a test case which uses in first step "Add(1)" another one which uses in the second step "Add(2)" and finally a test case which uses "Add(1)" in step 3 but not necessarily one single test case with the exact combination. You find the answers (as really a lot of question were asked there) in the forum and help of Spec Explorer:

http://msdn.microsoft.com/en-us/library/ee620427.aspx

http://social.msdn.microsoft.com/Forums/en-US/977b90c1-8938-474a-840e-14fd78b1af3e/spec-explorer-wmethod?forum=specexplorer

Spec Explorer is used in real world testing so the problem (only one of the many in MBT) with exponential explosion for path coverage had to be worked around. The extremely cool solution of Spec Explorer is the Cord-language (or regular language if you want). Instead of tedious programming test cases Spec Explorer allows us now to only sketch the test case with scenarios. The details and combinations comes out of the generic model. In practice this is what we (at least all the projects I did) really want. And as you see you can add your missing test case if you really need it.

goofy
  • 549
  • 2
  • 7
  • Thank you for your answer. I already expected something like this, but because several other transition where double covert I was doubting myself. I couldn't find their strategy anywhere, so I was worried that my cases just randomly vanished. The links are very helpful so thank you. – Kyra Sep 18 '14 at 07:04