0

I wonder if there is any tool that automatically generates unit tests based on a method's pre- and post-conditions (and class invariant).

To be precise, suppose we are given pre- and post-conditions (and probably class invariant) for a method and we want to generate unit tests based on that information. I am looking for a practical tool for this purpose.

I know Korat and TestWizard. Could you please name a few more techniques/tools?

PS: Programming language, or the language using which we specify the method contracts, is not that important. I just want to know about the techniques.

Thank you.

RealNamesOrHandles
  • 17
  • 1
  • 6
  • In the .net world, Pex used to do that but I don't know if it's still maintained: https://www.microsoft.com/en-us/research/project/pex-and-moles-isolation-and-white-box-unit-testing-for-net/?from=http%3A%2F%2Fresearch.microsoft.com%2Fpex%2F – vc 74 Feb 03 '19 at 07:29
  • @vc74, thanks for mentioning Pex:) I think Pex generates tests based on parameterized test methods, not based on method contracts. If you are sure that they have added what I am looking for, please let me know. – RealNamesOrHandles Feb 03 '19 at 07:41
  • It does generate tests based on code contracts too although documentation on this seems to be hard to find these days. Here's an example: https://blogs.u2u.be/peter/post/Pex-and-Code-Contracts – vc 74 Feb 03 '19 at 07:45
  • Thanks for the link; I was not aware of that. But it still does not do what exactly I want. Based on what I understood from the article you gave as example, code contracts are not used to _generate_ test cases, instead they are used to guide Pex to avoid non-sensical inputs. – RealNamesOrHandles Feb 03 '19 at 07:59

2 Answers2

0

Randoop does that. Its manual tells you how to specify pre- and post-conditions.

In a comment you made a distinction between "generating" and "guiding"; I'm not sure exactly what you mean by that, but you might view Randoop as using the specifications for guidance.

mernst
  • 7,437
  • 30
  • 45
  • Thank you professor Ernst for mentioning this (new?) feature of Randoop:) It will definitely help me, esp. because Randoop is a robust tool. Regarding your doubt about my comment, I would say the story is completely different in Randoop. Pex requires the user provide a PUT and then uses the code contracts to guide test case generation (probably by providing more info. to the underlying SMT solver). But Randoop does not need PUT and is much closer to what I am looking for. – RealNamesOrHandles Feb 03 '19 at 18:26
0

One of research activities in this area was carried out in 2005-2011 in ETH to make a "Contract-based random testing tool", aka AutoTest. It was developed for Eiffel where preconditions, postconditions and class invariants are part of the language. The tool uses them as a specification to generate tests automatically. The tool was integrated with EiffelStudio development environment where it employs contracts for 3 kinds of tests:

  • manual - a good "old-school" approach by writing tests by developers;
  • extracted - as soon as there is a contract violation, the tool can generate the corresponding test case;
  • generated - the tool synthesizes new tests automatically by making sure preconditions and class invariants are satisfied on entry to a routine and checking that there is no failure on return, in particular, that there is no postcondition or class invariant violation.
Alexander Kogtenkov
  • 5,770
  • 1
  • 27
  • 35
  • Thanks Dr. Kogtenkov:) It will definitely help me, sir. It's been a long time since I read AutoTest paper, but I think we can think of it as "Randoop for Eiffel," right? – RealNamesOrHandles Feb 03 '19 at 18:58
  • I do not know the details, e.g. whether Randoop supports "extracted" tests. On the front-end side, the contracts [known to Randoop](https://randoop.github.io/randoop/manual/index.html#kinds_of_errors) are a bit limited, whereas Eiffel code comes with extensive contracts from the beginning. Historically, the research was performed more or less in the same time frame. I have no idea whether and how both groups influenced each other. – Alexander Kogtenkov Feb 03 '19 at 20:12