17

I'm really at loss on what's going on - it all begun with PHPUnit error of Error: No code coverage driver is available when trying to run test coverage report and ended with me debugging to a replicable set described below. But to set the stage - I'm using Laravel 5.5, Xdebug 2.5.5, PHPUnit 6.5.5. My test code that illustrates the issue:

<?php

use Tests\TestCase;

class A extends TestCase
{
    public function testA()
    {
        echo( get_cfg_var('cfg_file_path')); exit;
    }
}

outputs C:\Users\xxx\AppData\Local\Temp\7598.tmp

compare it to this code which outputs the correct php.ini path:

<?php

use PHPUnit\Framework\TestCase;

class A extends TestCase
{
    public function testA()
    {
        echo( get_cfg_var('cfg_file_path')); exit;
    }
}

outputs: C:\server\php\php.ini

How can it be? How can loaded php.ini file change dependant upon the executed code? Better yet - how can my correct php.ini file (that has xdebug enabled) be loaded, instead of this imposter?

In both cases tests are launched using phpunit tests\unit\a

Folder structure is:

Laravel Project
└───tests
    └───Unit
        └───A.php
eithed
  • 3,933
  • 6
  • 40
  • 60
  • **How exactly are you launching your tests?** Test frameworks can often use custom `php.ini` file, and some do that by default. PHP interpreter [allows it](http://php.net/manual/en/features.commandline.options.php). Is it possible that's what's happening here? – Smuuf Jan 07 '18 at 23:27
  • In both of the situations I'm launching it using `phpunit` command. In my understanding, if custom php.ini were to be loaded, it would be loaded in both of the cases – eithed Jan 08 '18 at 08:09
  • So the only difference here is different class path in the `use` statement? – Smuuf Jan 08 '18 at 10:48
  • Yup!___________ – eithed Jan 08 '18 at 11:51
  • So, in that case, what exactly does `Tests\TestCase` extend? Does the hierarchy go back to `PHPUnit\Framework\TestCase`? – Smuuf Jan 08 '18 at 12:53
  • `Tests\TestCase` is the default TestCase provided by Laravel (https://laravel.com/docs/5.5/testing) – eithed Jan 08 '18 at 14:26
  • TBH: no idea. Test project with Laravel 5.5.19; PHPUnit 6.4.3; PHP 7.1 nts 32-bit on Windows 10 x64; xdebug 2.5.5. Executing `.\vendor\bin\phpunit ./tests/Unit/ExampleTest.php` from the project root folder (will test default sample test file) .. and it shows correct `php.ini`. Maybe some PHPUnit settings? – LazyOne Jan 08 '18 at 15:51
  • @LazyOne - I can confirm that testing a **clean** installation of Laravel doesn't replicate the issue. I've overwritten project's `phpunit.xml` with one from clean installation and the issue still exists. – eithed Jan 08 '18 at 16:29
  • @eithed Project-specific case .. heh (where, in case of a bug, dev may say "don't know... works on my computer"). Anyway -- check dependencies then: try installing the same additional packages that you have in your main composer.json, but 1 by one -- it may appear at some point. Check what classes/service providers are loaded etc. Maybe do some global project search for some settings or process related functions (that can make such change), even functions that call temp file/folder creations. Really -- no better clues/solid ideas. – LazyOne Jan 08 '18 at 16:41
  • @eithed Another thought though: try running that "bad" test with xdebug profiler -- it should list all stuff that will be called .. so may give you some clues/ideas if you can search trough that of course (although the number of called functions will be HUGE in your case -- lots of framework/library code gets called). – LazyOne Jan 08 '18 at 16:43
  • @LazyOne - sounds good (given that colleague of mine was saying that Voyager is at fault), though - can you explain your reasoning behind this? Afaik - once PHP is running, php.ini has already been loaded, so how could executed code modify this behaviour? The second option sounds good as well - will give it a try, thank you! – eithed Jan 08 '18 at 17:58
  • Could you please provide your project folders structure and exact commands you execute (including the folder full path you run those tests from)? – Alex Jan 10 '18 at 15:45
  • @Alex - the command is in the OP; I'll add the folder structure, but I'm not doing anything weird, so it can be deduced from the command :) – eithed Jan 10 '18 at 18:04
  • `phpunit tests\unit\a` is just one command, and there is no 2nd one in OP, and there is no info about the folder from which this command is called. You can be 100% right, but from my experience I had situations when many php, phpunit , laravel instances/versions lived on the same machine and it was not that easy to catch some weird bugs. Good luck anyway with your investigation – Alex Jan 10 '18 at 18:20
  • @Alex - it's one command as it's the same test that's being called; I'm running test, observing the result, changing the `use` within the test and running the command again – eithed Jan 10 '18 at 19:46
  • oh, wow! got it! – Alex Jan 10 '18 at 19:50

2 Answers2

11

We've tracked it down to an issue with Composer's XdebugHandler.php::writeTmpIni function located in vendor\composer\composer\src\Composer\XdebugHandler.php.

Apparently during App initialization a separate php process is spawned with temporary php.ini, and to that php process is the test passed to, but why it is done so is currently beyond me.

Will flag it up on Laravel's bugtracker in regards to what to do with this.

The dependancy that adds Composer, as a package (in my case) is larapack/hooks, which in itself is a dependancy for larapack/voyager-hooks, which in itself is a dependancy for Voyager.

As far as a I know, during the initialisation of the Laravel application this behaviour shouldn't be triggered (why initialise a dependency that is not required, at least explicitly). Why does Composer trigger itself at that stage is beyond me as well.

The solution we've applied is to add:

<php>
    <env name="COMPOSER_ALLOW_XDEBUG" value="1"/>
</php>

in the phpunit.xml file


I've now submitted this as an issue: https://github.com/laravel/framework/issues/22782


Per discussion on GitHub it's caused by the change in Laravel 5.5, in regards to handling service providers (https://laravel.com/docs/5.5/packages#package-discovery). How it gets updated, I don't know - to me it's a change between 5.4 and 5.5 that deserves a note in the upgrades (but please, read the discussion that happened on Github and make your own decision about the matter); to be honest, reporting this issue left a sour taste in my mouth and I won't be pursuing it further.


I've also opened an issue on larapack/voyager-hooks in regards to handling service providers discovery introduced in 5.5 - https://github.com/larapack/voyager-hooks/issues/16

This has now been fixed in larapack/hooks:v1.0.3

eithed
  • 3,933
  • 6
  • 40
  • 60
  • (random thought) It could be the performance optimisation (e.g. to alter the config to disable xdebug extension) .. but that may have any sense of composer's own functioning (e.g. `composer update` or alike) and not for app usage... Would be good if you post back your findings with Composer (as additional comment mentioning me) - I would like to know the outcome (quite interesting situation overall). – LazyOne Jan 08 '18 at 19:27
  • No worries - will do. What I've found interesting is that the clean Laravel projects don't have `vendor/Composer` inside the packages, so it might be an out-of-date dependancy. Still - have to go through all the packages, check where it's coming from, and if it indeed has been an issue (with previous versions of Composer) / does it exist in current version (doubtful, seeing as clean installations don't have these problems). Composer does complain about xdebug running, but it's the first time I've seen something as peculiar as this. – eithed Jan 08 '18 at 19:43
  • What I would do is to put a breakpoint in the `writeTmpIni()`, or maybe just throw an exception there, so I could see the stack trace and see _why and by whom_ there's `composer` involved in running tests. *(I would understand it, if the tests were executed by Composer's `composer exec phpunit -vvv`, but it seems like that's not the case.)* – Smuuf Jan 08 '18 at 23:13
  • @Smuurf that's certainly a good idea to help with tracking. What I've done so far is searched the `vendor` folder for all `composer.json` containing requirement of `composer` and there was only one instance of that (in `larapack/hooks`), so that seems to be the culprit. – eithed Jan 09 '18 at 11:22
1

I had some other issues seemed weird to me before. It seem you are using Windows machine as your dev. I am not sure about your php installation. But what happened to me I had a lot of different instances of apache, php, phpunit, nginx, mysql located all around different folders.

My point is it looks like you have many php installed on your machine. And for some reason different php interpreters are in use for 2 cases described in OP.

I can be wrong and still have no response on my comment about commands and folders you use to execute both tests.

But you can just output

echo PHP_BINDIR;

in both tests to be sure that same php interpreter is in use in both cases.

Alex
  • 16,739
  • 1
  • 28
  • 51
  • The same php interpreter is executed in both cases. The problem is that when `use Tests\TestCase;` is used, then the Laravel app is initialized, which spawns a new php request, which uses a dynamically generated php.ini that has xdebug commented out. Of course if you use `use PHPUnit\Framework\TestCase` then Laravel app is not initialized, which doesn't produce the issue. – eithed Jan 10 '18 at 18:01