Good way to repeat a test, inserting an extra action?

Question

I like the way Catch has nested hierarchies of tests, and it works through the combinations. It feels more natural than the setup/teardown of xUnit frameworks.

I now have a set of tests. What I want to do, about halfway down is insert a load/save serialization test, and then repeat all the tests below that point, first without the load/save, then again using the data it loaded from the serialization process. I.e. to prove that the load/save was correct.

I cannot get my head around if Catch has anything that can help with this? If it was phpUnit, I would be thinking about a string of @depends tests, and use a @dataProvider with a boolean input. A bit ugly.

(If that does not make sense, let me know, and I'll try to work out a minimal example)

JBRWilkinson · Answer 1 · 2015-07-09T20:19:11.093

The issue here is that Catch is designed to descend a tree-like organisation of tests and it automatically discovers all of the leaf-nodes of the structure and calls back into the test cases with untested code paths until they're all tested. The leaf nodes (tests, sections) are meant to be independent.

It sounds like you want to test a repository - something that can persist some data and then load it back in.

To repeat the exact same tests in two different scenarios (before serialisation, after serialisation) you'd need to put the same tests into some common place and call into that place. You can still use the same Catch macros in a non-test-case function, as long as you call it from a test case.

One possible way to do this is:

struct TestFixture {
  Data data;
  Repository repository;
  TestFixture() : data(), instance() { }
};

void fillUpData(Data& data) {
  // ...
}

void isDataAsExpected(Data& data) {
  // Verify that 'data' is what we expect it to be, whether we
  // loaded it or filled it up manually
  SECTION("Data has ...) {
    REQUIRE(data...);
  }
}

TEST_CASE_METHOD(TestFixture, "Test with raw data") {
  fillUpData(data);
  isDataAsExpected(data);
  REQUIRE(repository.save(data));
}

TEST_CASE_METHOD(TestFixture, "Operate on serialised data") { 
  REQUIRE(repository.load(data));
  isDataAsExpected(_data);
}

One possible alternative is to supply your own main and then use command-line arguments to control whether/not the data is first serialised.

There's a third way I can think of that uses a non-quite-ready-yet feature of Catch - Generators:

TEST_CASE("...") {
  using Catch::Generators;
  int iteration(GENERATE(values(0, 1)));
  const bool doSave(iteration == 0);
  const bool doLoad(iteration == 1);
  Repository repository;
  Data data;
  if (doLoad) {
    REQUIRE(repository.load(data));
  } else {
    // fill up data
  }

  REQUIRE(..data..test..);

  if (doSave) {
    REQUIRE(repository.save(data));
  }
}

The advantage of this method is you can see the flow and the test runs twice (for the two values) but the major disadvantage is that Generators are incompatible with SECTIONs and BDD-style features.

Thanks for the ideas. This is not exactly what I wanted to do, so I guess I haven't explained the question very well. I will try to come back to it with some sample code. (It has also been a couple of months, so I'll have to go check those tests, to see if I came up with my own solution!) — Darren Cook, Jul 09 '15 at 20:47

Good way to repeat a test, inserting an extra action?

1 Answers1