Simultaneously running 2 or more boost testcases belonging to different test suites via cmd

Question

Consider the following scenario:

BOOST_AUTO_TEST_SUITE(suite1)
{
    BOOST_AUTO_TEST_CASE(case1)
    {
        //my test code here
    }
}

BOOST_AUTO_TEST_SUITE(suite2)
{
    BOOST_AUTO_TEST_CASE(case1)
    {
        //my test code here
    }

    BOOST_AUTO_TEST_CASE(case2)
    {
        //my test code here
    }
}

Now, if I want to run suite1/case1 and suite2/case2 at once, I try the following command line argument:

MyProject.exe --run_test="suite1/case1, suite2/case2"

But this doesn't seem to run.

I know that I can separately run these test cases, as:

MyProject.exe --run_test="suite1/case1"

and

MyProject.exe --run_test="suite2/case2"

But I want to run them together at one go. What should I do? Thanks in advance :)

Have you tried omitting the space between the test names: "suite1/case1,suite2/case2" ? — oakad, Apr 14 '14 at 05:50
@oakad Doesn't work! Error: **"Test setup error: no test cases matching filter"** — CinCout, Apr 14 '14 at 06:21
I have tried this stuff on my side and it feels like there's a bug in the boost.test suite handling code (after all, multiple tests can be invoked when suites are not involved). May be you should try filing a bug report in boost trac system. — oakad, Apr 14 '14 at 07:01
Just the usual doc file: http://www.boost.org/doc/libs/1_55_0/libs/test/doc/html/utf/user-guide/runtime-config/run-by-name.html. More specifically, you may notice that `testA` and `testB` used for the multiple tests example are outside of any suite definitions (I have tried doing this to more than 2 tests, it does work). — oakad, Apr 14 '14 at 09:06
What I meant was a tutorial to file the bug! Anyways, I've already seen this doc, and the use case I'm referring to doesn't exist here! — CinCout, Apr 14 '14 at 10:50
Mmm. Perhaps I took the 'running simultaneously' a bit literal there. Wording question titles is an art :) — sehe, Apr 19 '14 at 13:23

score 3 · Answer 1 · answered Apr 18 '14 at 18:59

This is not a feature currently supported by Boost.Test. The documentation states that you can use a comma separated list if the tests are in the same suite:

Running multiple test cases residing within the same test suite by listing their names in coma separated list.

You can also use wildcards to select suites and test cases, but depending on the names of your suites and cases, you may not be able to limit the selection to just to two cases you desire.

http://www.boost.org/doc/libs/1_55_0/libs/test/doc/html/utf/user-guide/runtime-config/run-by-name.html

sehe · Answer 2 · 2014-04-19T13:37:13.427

Edit It seems I might have taken the question title a bit too literally. Running tests simultaneously means "in parallel" to me.

Anyways, if you are happy to run suite2/case1 as well, you can just

MyProject.exe --run_test="suite1,suite2"

See it Live On Coliru too.

Old answer: What is wrong with running the two processes in parallel? By all means, uncomplicate!

However, if you insist, you can fork copies of the main process:

#include <sys/types.h>
#include <sys/wait.h>
#include <iostream>

static int relay_unit_test_main(std::vector<std::string> args);

int main()
{
    if (int const child_pid = fork())
    {
        int exit_code = relay_unit_test_main({"--run_test=suite1"});

        int child_status;
        while (-1 == waitpid(child_pid, &child_status, 0));
        if (!WIFEXITED(child_status)) {
            std::cerr << "Child process (" << child_pid << ") failed" << std::endl;
            return 1;
        }

        return exit_code? exit_code : WEXITSTATUS(child_status);
    } else
    {
        return relay_unit_test_main({"--run_test=suite2"});
    }
}

See it Live On Coliru

The function relay_unit_test_main is really nothing more than a convenience wrapper around unit_test_main that avoids meddling with argv[] manually:

static bool init_function() { return true; }

static int relay_unit_test_main(std::vector<std::string> args)
{
    std::vector<char const*> c_args;
    c_args.push_back("fake_program_name");
    std::transform(args.begin(), args.end(), std::back_inserter(c_args), std::mem_fn(&std::string::data));
    c_args.push_back(nullptr);

    return unit_test_main( &init_function, c_args.size()-1, const_cast<char**>(c_args.data()) );
}

This actually spawns a child process - and even tries to usefully combine the exit code information. Having a separate process prevents the problems that you'd get from using code that wasn't designed for multi-threaded use on different threads.

One caveat remains: if your program does static initializations before entry of main(), and these use external resources (like, log files, e.g.) there might be conflicts. See

man fork(3)
Does Boost Log support process forking? for an example of a lib that has potential issues with fork()

Edited my answer to give a workaround for the (much simpler) question :) Note that you can still use the `relay_unit_test_main` hack, perhaps with `boost::program_options` to achieve your actual goal. However, it would seem more reasonable to "just" write a patch against Boost Test and contribute that! — sehe, Apr 19 '14 at 13:32

Simultaneously running 2 or more boost testcases belonging to different test suites via cmd

2 Answers2