How Best to Toggle specific Fitnesse Tests?

Question

Can specific fitnesse tests be toggled on or off conditionally?

In Fowler/Hodgson terminology, we are using "Release Toggles" - i.e. toggles which allow changes to be disabled in our test and pro environment until ready for prime time. We don't want failures in these features to affect the shipment to pro so aren't interested in the results of those tests. In that sense, the CDP pipeline has the toggles set to "PRO" configuration all the way through to keep the acceptance tests fast and focused. So how to get fitnesse to ignore the tests related to the toggle?

I see this related answer: How should feature toggles be set in tests run in continuous integration? which indicates that this user tests all sides of the toggles, but this is not usually possible for us - e.g. imagine that a feature requires an application restart to change the toggle due to DI or whatever (e.g. because the feature requires wiring a different web controller) - this is a pretty hard thing for fitnesse to do.. makes for slow tests and is testing functionality we don't actually need to test (yet).

So we want to be able to develop the feature (behind a toggle) and the tests, fixtures (marked with the same toggle) and when we are ready to enable them flick them on to run through the CI pipeline. Something like:

1. Decide feature needs a toggle so create one TOG123-change-api-behaviour
2. Add the toggle to the central toggle system which all code can see
3. Set toggle to "false" in configuration for CI environments
4. Develop feature and fitnesse tests behind toggling code
5. When ready to CDP set to true
6. When passing test, ship to pro
7. in time.. remove the toggle by literally searching for TOG123 and cleaning up the mess..

The bit I don't see a good answer for is how to guard specific fitnesse tests with conditions (i.e. the new test would have a when: TOG123.enabled and the old test would have a when: TOG123.disabled). Fitnesse seems a tad.. awkward to me. Are tags what I'm looking for? Suites seem too broad a level..

Answer 1

Based on your description Tags seems to be what you are looking for. They allow you to mark your tests for the new feature and choose to include or exclude those tests in a run you execute.

On a side note: if the features are real 'toggles' i.e. they can be enabled in production without requiring a new version, I would recommend testing all sides of the toggles; otherwise you don't know whether enabling the toggle is actually a good/safe thing to do. If the new features are not expected to be 'production ready' and therefore do not warrant testing before deployment: why include them in your product version being released? In that situation I would recommend you keep those new features in a separate branch, which is not yet merged to development/master. You can then maintain/develop the tests for those feature in the same (or similar, if the tests are in a different repository). This prevents complexity/code in production which should not be used and also prevents the need to 'clean up the mess' afterwards: the cleanup is part of the branch. When the branch is merged the new functionality is added, tested and now outdated tests/code is immediately removed. This of course does require the ability to run/test your system based on a branch, but that is something you have when doing continuous delivery.