9

In my project, I have several QuickCheck properties, most of which I collect using forAllProperties, defined in Test.QuickCheck.All. I am trying to run all my properties in parallel which is proving troublesome: at the end of a run, I get the output printed in the terminal and counterexamples and property names are often scattered so as to make it difficult to match properties with their counter examples.

I see that the purpose of library pqc is to run properties in parallel but it does not provide a replacement for forAllProperties nor does it provide a way of combining forAllProperties with the parallel test driver.

It feels like all I would need is for forAllProperties to pass the property name to the function it gets as an argument.

I have also looked into redirecting stdout on a thread by thread basis, first by using system-posix-redirect (which is not thread safe), then by studying Test.QuickCheck.State, especially the terminal field. The latter didn't pan out because I did not find a way of rewriting the terminal field.

Is there a way for me to, somehow, output the counter-examples together with the property names without copy / pasting the Test.QuickCheck.All module and making the changes I need?

simon1505475
  • 675
  • 3
  • 9
  • Would something like `$forAllProperties (\x -> p{Det/Non} x k)` where `k` is some depth work for you? – user2407038 May 13 '16 at 23:10
  • I believe not. Please correct me if I'm wrong but running two such statements in parallel still gets the putStrLn in $forAllProperties to appear out of order with the counter examples. So I get stuff like: – simon1505475 May 15 '16 at 14:26
  • `=== prop_stuff in Foo.hs ===` `=== prop_fooBarFoo in Bar.hs ===` `Failed after 7 tests:` `7` `7 /= 3` `Failed after 3 tests` `3` `3 /= 4` So it is hard to match the counter examples to the properties. Even more so with thousands of individual properties (gathered with forAllProperties). – simon1505475 May 15 '16 at 14:44
  • `quickCheckWithResult` returns the `String` that is printed in the `output`. – ja. May 16 '16 at 17:17
  • Looking at the source (http://hackage.haskell.org/package/QuickCheck-2.8.2/docs/src/Test-QuickCheck-All.html#quickCheckAll), it seems it will print property name header form main thread no matter what you supply to `forAllProperties`. So copy-pasting `Test.QuickCheck.All` seems like the way to go. – lierdakil May 17 '16 at 17:57
  • 3
    You ask, "Is there a way to do X without making changes to Test.QuickCheck.All?". Why should we rule out that solution? It seems the natural one to me; and if there is a better one, then *both* should be listed here, and the better one upvoted more! – Daniel Wagner May 18 '16 at 02:14
  • @DanielWagner: you're right, it is one possible solution. I was using it as a baseline to compare other solutions. I don't like it because it requires duplicating library code, which seems like a last resort solution to me. I ended up doing it anyway but I wish there was a way around it. Maybe I should write to the authors and donate the code I ended up with. – simon1505475 May 18 '16 at 21:04
  • @ja: that's correct but that string does not contain the property's name. What I would really like is a way to use Result to get both the `output` and the name of the property. – simon1505475 May 18 '16 at 21:05
  • @lierdakil: I am unclear about the relation between your two sentences. Do you mean to point out that `forAllProperties` does have access to the property name so copy pasting it will let me recombine the names and the Result however I please? – simon1505475 May 18 '16 at 21:08
  • Sorry about being unclear. I was alluding to last paragraph in your question, specifically "copy / pasting the Test.QuickCheck.All module and making the changes I need" -- to me this seems like the only option, unless you manage to somehow trick TH to use custom version of `runQuickCheckAll`, which is most likely impossible. – lierdakil May 19 '16 at 00:45
  • 1
    I think what others are getting at is that the best solution is to modify Test.Quickcheck.All *in place*. It's right over there --> https://github.com/nick8325/quickcheck , I'm sure they would be accepting of Issues or PRs. – chreekat Oct 20 '16 at 20:54

0 Answers0