4

What is the best way to tuple: 1. QuickCheck inputs to a predicate with 2. the outputs returned by a tested function ?

Wanting to test (2 * 350) functions in a couple of other languages (to check for divergence from the behaviour of the Haskell base and Prelude functions which they imitate) my first experiment has been to:

  • capture output from (QuickCheck) verboseCheck result strings
  • apply the Haskell function to each of the (re-read) inputs which QuickCheck threw at it, to obtain the corresponding Haskell output
  • use these inputs and outputs in automated checks of the parallel functions in other languages for divergences from the behaviour of the Haskell function

For example, to get test data for a function on a list, something like:

testSet1
  :: (Show a, Testable prop)
  => String -> String -> ([Int] -> a) -> prop -> IO ()
testSet1 folder fname f p = do
  r <- verboseCheckResult p
  let ys =
        ((,) <*> f) . read . listTuple1 <$>
        splitOn ["Passed:"] (tail $ init $ (lines . output) r)
  writeFile (concat [folder, fname, ".txt"]) (show ys)

This works OK, at least for simple cases, but I cannot immediately find ways of generalising this kind of pattern to avoid having to write a separate function (to generate and capture inputs with outputs) for every variant of type signature.

The key problem is that read typically needs some type information with which to parse the list of QuickCheck-generated inputs from their (QuickCheck result output) stringifications.

Perhaps I am taking the wrong approach. Any thoughts ? Template Haskell ?

houthakker
  • 688
  • 5
  • 13
  • It appears you want to reduce code duplication. It may be illustrative to add a second example so one can understand the problem just from reading code. Imagine how you might pose the question to the smartest possible automated refactoring tool that does not understand natural language. – Gurkenglas Feb 01 '18 at 12:09
  • Probably you may write a sufficiently generalized helper that explicitly generates a list of inputs (see the `Gen` typeclass) for a given function, then feeds and compares it without calling `quickCheck` itself. – Yuuri Feb 01 '18 at 12:13

1 Answers1

2

There is a separate package to annotate properties with counterexamples (which can consist of inputs and outputs): quickcheck-with-counterexamples.

This QuickCheck issue on Github has some discussion on the same topic.

Li-yao Xia
  • 31,896
  • 2
  • 33
  • 56