How display name of failed PHPUnit test even before all tests finish

Question

I've split my tests into different groups so that I don't bother running the slowest ones until absolutely necessary.

However, some are still slow.

So I'd love to see the names of tests that either failed or had errors even before the whole suite of tests finishes running.

E.g. currently when I run phpunit tests/Unit/ --verbose, I see output like this:

PHPUnit 7.5.1 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.2.14-1+ubuntu16.04.1+deb.sury.org+1 with Xdebug 2.6.0
Configuration: /home/vagrant/Code/myproject/phpunit.xml

........F.....

But even though I see the "F", I need to wait a long time before I see which test it was that failed.

I'd love to be able to allow the tests to continue running but simultaneously start looking into that particular test to see why it failed.

How?

Answer 1

You have at least 3 options:

1. add --debug flag (after reading your comments it seems this won't do what you need which is show in real time what test failed)

2. create a printer class and use it with --printer flag

3. add --testdox flag

A long time ago I had to do my own printer because I needed to display how long a test took. So basically I was printing when a test started, its name and how long it took. You can use this code and adapt it to your needs.

You need to create a new class (usually under /tests directoy). Make sure this class Extends PHPUnit_TextUI_ResultPrinter. Then in your phpunit.xml file you add these settings inside

     printerFile="tests/TestDurationPrinter.php"  <-- filename 
     printerClass="TestDurationPrinter   <-- Class you created.

My Printer looks like this:

    class TestDurationPrinter extends \PHPUnit_TextUI_ResultPrinter
    {
    /**
     * @param PHPUnit_Framework_Test|PHPUnit_Extensions_PhptTestCase $test
     */
    public function startTest(PHPUnit_Framework_Test $test)
    {
        $this->write(sprintf(
            "\nRunning '%s::%s'...",
            get_class($test),
            $test->getName()
        ));
    }

    /**
     * @param PHPUnit_Framework_Test|PHPUnit_Extensions_PhptTestCase $test
     * @param float $time
     */
    public function endTest(PHPUnit_Framework_Test $test, $time)
    {
        $this->write(sprintf(
            "ended and took ~ %s seconds.",
            round($time)
        ));
    }
}

In your case, I think using --testdox would be enough. If it doesn't work, then I'd recommend creating your own printer, bear in mind that you still need to get access to the test result (which I don't know how to do it) so you would need to google that.

I hope this helps you.

Edit: another thing you can do is to add the flag

--stop-on-error

so that tests stop as soon as there's an error. There's also

--stop-on-failure

(I think this is want you want). I always use both as my testsuite has around 800 tests and the entire execution takes around 3 minutes.

Answer 2

I appreciated Guillermo Mansilla's answer, and now I've finally come up with something I prefer even more:

1) I edited the <phpunit> tag of my phpunit.xml to include:

printerFile="tests/CustomPhpUnitConsolePrinter.php"
printerClass="CustomPhpUnitConsolePrinter"

2) My CustomPhpUnitConsolePrinter.php is this:

<?php

class CustomPhpUnitConsolePrinter extends \PHPUnit\TextUI\ResultPrinter {

    protected static $skippedTests = [];
    protected static $testsWithUnknownStatus = [];

    /**
     * @param \PHPUnit\Framework\Test|\PHPUnit\Extensions\PhptTestCase $test
     * @param float $time
     */
    public function endTest(\PHPUnit\Framework\Test $test, $time): void {
        parent::endTest($test, $time);
        $status = $test->getStatus();
        if ($status != \PHPUnit\Runner\BaseTestRunner::STATUS_PASSED) {//https://stackoverflow.com/a/6744246/470749
            if ($status == \PHPUnit\Runner\BaseTestRunner::STATUS_SKIPPED) {
                self::$skippedTests[] = \PHPUnit\Util\Test::describeAsString($test);
            } else if ($status == \PHPUnit\Runner\BaseTestRunner::STATUS_UNKNOWN) {
                self::$testsWithUnknownStatus[] = \PHPUnit\Util\Test::describeAsString($test);
            } else {
                $this->write(\sprintf(PHP_EOL . "[%s]", \PHPUnit\Util\Test::describeAsString($test)));
            }
        }
    }

    protected function printFooter(\PHPUnit\Framework\TestResult $result): void {
        if (self::$skippedTests) {
            $this->write(PHP_EOL . strtoupper('► Skipped tests:') . PHP_EOL);
            $this->write(implode(PHP_EOL, self::$skippedTests) . PHP_EOL);
        }
        if (self::$testsWithUnknownStatus) {
            $this->write(PHP_EOL . strtoupper('► Tests with Unknown status:') . PHP_EOL);
            $this->write(implode(PHP_EOL, self::$testsWithUnknownStatus) . PHP_EOL);
        }
        parent::printFooter($result);
    }

}

3) I also added this section before the closing </phpunit> tag (see docs):

<logging>
    <!--        https://stackoverflow.com/a/52377685/470749-->
    <log type="testdox-html" target="./storage/logs/testdox.html"/>
</logging>

Answer 3

Just one tiny detail with @ryan's own answer. You can't extend an interface. What I did what to extend, instead,

PHPUnit\TextUI\DefaultResultPrinter