Test pass just when there's a single test method

Question

If I comment out one of the tests bellow the test passes. However, running both together, the last one will fail (even if I change tests order):

Production code:

<?php
class View
{
    private $filename;
    private $data;

    public function __construct($filename)
    {
        $this->filename = $filename;
        $this->data = [];
    }

    public function __set($key, $value)
    {
        $this->data[$key] = $value;
    }

    public function render()
    {
        extract($this->data);
        ob_start();
        require_once $this->filename;
        return ob_get_clean();
    }

    public function __toString()
    {
        return $this->render();
    }
}

Test class:

require_once 'vendor/autoload.php';

use org\bovigo\vfs\vfsStream;
use org\bovigo\vfs\vfsStreamWrapper;
/**
 * @outputBuffering enabled
 */
class ViewTest extends PHPUnit_Framework_TestCase
{
    public function setUp()
    {
        vfsStream::setup();
    }

    /**
     * @outputBuffering enabled
     */
    public function testRenderSimpleView()
    {
        $fileContent = 'index file';
        vfsStreamWrapper::getRoot()->addChild(
            vfsStream::newFile('index.php')->withContent($fileContent)
        );
        $view = new View(vfsStream::url('index.php'));
        echo $view->render();
        $this->expectOutputString($fileContent);
    }

    /**
     * @outputBuffering enabled
     */
    public function testRenderViewWithData()
    {
        $filename = 'index.php';
        $fileContent = '<?php echo $a; ?>';
        vfsStreamWrapper::getRoot()->addChild(
            vfsStream::newFile($filename)->withContent($fileContent)
        );
        $view = new View(vfsStream::url($filename));
        $view->a = 1;
        echo $view;
        $this->expectOutputString('1');
    }
}

test output:

PHPUnit 3.7.10 by Sebastian Bergmann.

.F

Time: 0 seconds, Memory: 3.75Mb

There was 1 failure:

1) ViewTest::testRenderViewWithData
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'1'
+''


FAILURES!
Tests: 2, Assertions: 2, Failures: 1.

It doesn't make any sense to me. What am I missing?

Such behavior is usually due to issues with test isolation. Something from Test1 is not being cleaned up causing different behavior when Test2 runs after it. Are you using any shared/global variables or singletons ? Isn't the test failure/error/exception helpful? — Gishu, Dec 18 '12 at 09:08
No, I'm not using singleton or global data. Output Buffering affects PHP output behavior ... — Thom Thom Thom, Dec 18 '12 at 20:51

score 0 · Answer 1 · answered Dec 17 '12 at 23:18

Have you tried calling $view->Dispose() after executing the test ? I'm not a vfsStream expert, but it looks like the file will remain open between tests, and if the current file position is not rewound after reading it would remain at the end of the file, causing the failure on the second and subsequent tests.

score 0 · Answer 2 · answered Dec 21 '12 at 03:23

When that happens, it's usually because you're not isolating the method that you are testing properly. In other words, PHPUnit doesn't know you mean a different instance of a certain method that you are testing (or similar). Therefore, the second test always fails.

In cases where you use the same method more than once, you should use the "at" declaration with the proper count where executed in the code. This way PHPUnit knows which one you mean, and can fulfill the expectation/assertion properly.

The following is a generic example where method 'run' is used several times:

public function testRunUsingAt()
    {
        $test = $this->getMock('Dummy');

        $test->expects($this->at(0))
            ->method('run')
            ->with('f', 'o', 'o')
            ->will($this->returnValue('first'));

        $test->expects($this->at(1))
            ->method('run')
            ->with('b', 'a', 'r')
            ->will($this->returnValue('second'));

        $test->expects($this->at(2))
            ->method('run')
            ->with('l', 'o', 'l')
            ->will($this->returnValue('third'));

        $this->assertEquals($test->run('f', 'o', 'o'), 'first');
        $this->assertEquals($test->run('b', 'a', 'r'), 'second');
        $this->assertEquals($test->run('l', 'o', 'l'), 'third');
    }

Test pass just when there's a single test method

2 Answers2