handle tests using TAP::Harness, how to print output when test exits

Question

I am trying the whole day to find out the answer, but I didn't find anything. I wrote some tests using test::more (test1.t, test2.t, test3.t ...). and I wrote a main perl script (main.pl) that handles all the tests using TAP::Harness and print the output in a JUnit format using formatter_class => 'TAP::Formatter::JUnit. In my tests I use the BAIL_OUT function. The problem is that when a test is bailed out, the main script also exits and there is no output at all. If, for example test3.t bailed_out, I need to see the results for test1.t and test2.t. how can I do that?

I can't use exit or die instead of BAIL_OUT because I don't want the other tests to continue. (If test3.t was BAIL_OUT I don't want that test4.t will run.)

can someone please help me? I need to see the results for the tests that were running before the bailed out test.

Thanks.

fenway · Answer 1 · 2013-10-06T21:07:22.757

0

According to the Test::More docs:

BAIL_OUT($reason);

Indicates to the harness that things are going so badly all testing should terminate.
This includes the running of any additional test scripts.

So that explains why your suite aborts.

You might want to consider die_on_fail from Test::Most, or skip_all, depending on the reason for the BAIL_OUT.

EDIT: Looks like Test::Builder doesn't have any intention of printing out a summary when it exits on "catastrophic failure" according to the source code:

sub BAIL_OUT {
    my( $self, $reason ) = @_;

    $self->{Bailed_Out} = 1;
    $self->_print("Bail out!  $reason");
    exit 255;
}

# Don't do an ending if we bailed out.
if( $self->{Bailed_Out} ) {
    $self->is_passing(0);
    return;
}

However, that Bailed_Out flag is only ever used to consider printing out summary diagnostics, and since Test::More exposes the underlying Test::Builder object, you can probably just tweak the BAIL_OUT subroutine and no set this flag. All untested, of course; YMMV.

edited Oct 06 '13 at 21:07

answered Oct 06 '13 at 15:29

fenway

416
2
8

But I do need BAIL_OUT, because I do not need to run all other tests. if I use die or skip, other tests will continue running. I use BAIL_OUT when the test fail because the whole build failed and there is no reason to keep testing. It is ok that the suite aborts, but can I see the results for the tests that ran before I bailed out? – user1836185 Oct 06 '13 at 15:38
and I wish that only the relevant test suite will be aborted, why the main script with all previous test suits need to vanish as well? – user1836185 Oct 06 '13 at 15:47
That sounds nice, but how can I change the flag value? – user1836185 Oct 07 '13 at 07:00
And, it seems that even if I will change it to 'Flase' the program will still exit without printing the summery, am I wrong? – user1836185 Oct 07 '13 at 07:16

mikew · Answer 2 · 2013-10-07T21:20:18.333

Instead of passing in all tests to one TAP::Harness, you need to pass in one test at a time to the Harness in case of a BAIL_OUT

I haven't seen your code, so here is a sample of what I mean. Adjust to include the formatter and whatever else you need.

use TAP::Harness;

my $harness = TAP::Harness->new({ merge => 0 });
my $tests = ['t/test1.t', 't/test2.t'];

foreach my $test (@$tests) {
    eval {
        $harness->runtests([$test]);
    }; if ($@) {
        # create new harness object if the previous fails catastrophically.
        $harness = TAP::Harness->new({ merge => 0 });
    }
}

handle tests using TAP::Harness, how to print output when test exits

2 Answers2