4

I have a reasonably complex structure of data types and records, which is not so easy to make sense of by just looking at production code for someone not familiar with the codebase.

To make sense of it, I created this kind of dummy functions that have two advantages: 1. they're checked at compile-time and 2. they serve as some kind of documentation, showcasing an example of how the overall structure of data types and records mixes together:

-- benefit 1: a newcomer can quickly make sense of the type system
-- benefit 2: easier to keep track of how the types evolve because of compile-time checking
exampleValue1 :: ApiResponseContent
exampleValue1 = ApiOnlineResponseContent [
        OnlineResultRow (EntityId 10) [Just (FvInt 1), Just (FvFloat 1.5), Nothing],
        OnlineResultRow (EntityId 20) [Just (FvInt 2), Nothing, Just (FvBool True)]
    ]

The only thing that bothers me a bit is that they feel a bit awkward put within the production code, as they're clearly dead code. However they're not tests either, they're just compiled-time-checked examples of how values can be assembled together from complex nested types. Therefore, they clearly don't belong to production code, but they don't quite belong to tests either.

Is it common practice to have this kind of compile-time examples? And where should they be placed within the codebase?

Jivan
  • 21,522
  • 15
  • 80
  • 131
  • 1
    I like the style used by the [`Control.Monad`](https://hackage.haskell.org/package/base-4.15.0.0/docs/Control-Monad.html#v:mapM) documentation for this: an expandable section named "Examples". – Daniel Wagner Apr 30 '21 at 15:38

3 Answers3

4

Have you considered including the examples as part of Haddock documentation?

Otherwise, there's a school of thought that try to reframe tests as examples. You can find a brief mention of this in Gerard Meszaros's work on unit testing. Dan North has also repeatedly used similar language, but it can often be difficult to track down his many iterations of such ideas. He tends to 'think in public' - here's one such reflection:

"I call this development, using example-guided design"

I understand that the kind of example being asked about isn't quite like that. Still, I think that it better belongs with test code than with production code.

If you put the examples in the production code, you essentially make it part of the API of your library (if you're shipping a library). This means that changing the examples would constitute a breaking change. That doesn't seem right to me.

In the BDD/DDD community, there's a lot of emphasis on tests as examples, also in the sense that automated tests serve as documentation. If Haddock documentation isn't an option, I'd consider putting the examples in the test code, as documentation. I sometimes do that by simply putting such 'vacuous tests' in a file called examples, perhaps with a little comment at the top to explain that the code in that file assists learning rather than verify behaviour.

It dodges the risk of introducing redundant breaking changes in the production code, and seems conceptually like a better fit.

Mark Seemann
  • 225,310
  • 48
  • 427
  • 736
  • 1
    There's also doctest in a couple flavors, where tests are actually run on bits of documentation. – dfeuer Apr 29 '21 at 13:51
3

I disagree that these aren't tests. Even “does this example compile” could be seen as a test, but probably you could also use them to actually test some functions, while you're at it.

So, put these definitions in your test suite, and use them for unit testing the functions that will actually be dealing with such values.

leftaroundabout
  • 117,950
  • 5
  • 174
  • 319
1

The main convention I’ve seen for housing such examples is to include ….Tutorial modules, such as Dhall.Tutorial and Clash.Tutorial, or parallel …-tutorial packages such as lens-tutorial.

These contain Haddock documentation, organised so as to read in linear order, alongside example definitions like yours.

With good examples, I find this convention very helpful for understanding and experimenting with a new package in addition to types and reference docs. It’s also fairly discoverable when browsing packages on Hackage, especially if you ensure that the reference and README link to the tutorial for longer explanations.

These modules can be just compiled as basic validation, but having more machine validation for docs is incredibly valuable, so I think it’s even better to link them into the test suite (e.g. hspec) as examples for unit tests (HUnit) or property tests (QuickCheck/hedgehog), or organise them as documentation tests (doctest).

In addition to GHC’s code coverage tools, weeder can help identify examples that aren’t being tested.

Jon Purdy
  • 53,300
  • 8
  • 96
  • 166