multiple tests clashing in phing

Question

We have a Phing script that is used by Hudson to build/test our PHP site.

Some of our unit tests load the main function library; others use a mock to avoid having to do so (or to provide specific faked results for the test).

The unit tests all run perfectly when run in isolation (ie on the command line, using phpunit). However when we run them together as a batch in Phing, we get errors.

The errors are in tests where we have written mocks for certain functions. The error says that we are declaring the function twice. It is clearly attempting to include the real function library as well as the mocks.

The tests are including code that is mutually exclusive, so they need to be run in isolation from each other; it would appear that Phing is running them all in a single process, so the includes are clashing.

The relevant part of the phing script looks like this:

<phpunit haltonfailure="true" printsummary="true">
  <batchtest>
    <fileset dir="${ws}/path/to/site/root/">
      <include name="*Test.php" />
      <include name="*/*Test.php" />
      <include name="*/*/*Test.php" />
      <include name="*/*/*/*Test.php" />
    </fileset>
  </batchtest>
  <formatter type="xml" todir="${builddir}/logs" outfile="units.xml" />
</phpunit>

Is there a way to get phing to run the tests independantly of each other, without specifying each one separately in the build script?

By *when run in isolation* you mean `--process-isolation Run each test in a separate PHP process`? Do you have a phpunit.xml config file? — Gordon, Mar 18 '11 at 17:59
@Gordon - I suspect that Spudley would rather avoid running *each method* in a separate process and instead run all the mocked-library tests in one process and all the real-library tests in another. — David Harkness, Mar 18 '11 at 18:56
@Gordon: what I mean is that I don't want test 'X' to fail because it's trying to include a file that is `require()`d by test 'Y'. — Spudley, Mar 18 '11 at 18:57
So are you able to run the tests all at once using a single call to phpunit from the command line? Or are you using two calls, one for testing the real and one for testing the mocked stuff? Just to make sure i'll get the edit right — edorian, Mar 18 '11 at 19:17
@edorian: I'm not calling phpunit myself; it's just been set up in the phing config file to load all available tests (as per above). I don't know how phing works, but my guess is that it's loading them all into a single call to phpunit, because that's what seems to be causing the error. — Spudley, Mar 18 '11 at 21:39
You say in your question: "The unit tests all run perfectly when run in isolation (ie on the command line, using phpunit). However when we run them together as a batch in Phing, we get errors." - What i don't understand is if that is one cli call to phpunit or more. Sorry for cluttering the question with so many comments but i have a hard time understanding how your test suites look like it seems. Maybe you could drop by the php SO chat to get that sorted out? I'm going to update the question some more too — edorian, Mar 18 '11 at 22:04
@edorian: sorry for not being clear :) when I ran them from the command line, it was one at a time; ie `phpunit XTest.php`, followed by a separate `phpunit YTest.php`, etc. Sorry; can't drop by chat now... am off to bed! — Spudley, Mar 18 '11 at 23:46
@Spudley are there, by any chance, lots of require and include statements in your code? Can you remove those in favor of an autoloader? — Gordon, Mar 19 '11 at 08:57
@Gordon: The code that is clashing with our code is Drupal core. My test mocks a few of the Drupal core funcs to avoid having to include the whole of Drupal when testing my classes, and to provide fixed testable results. Some of our other tests do need to include the Drupal core, hence the clashes. — Spudley, Mar 19 '11 at 22:33

edorian · Accepted Answer · 2011-03-18T22:30:25.657

Well the easiest/fastest workaround seems to be "don't use the phing task but start using the 'real' phpunit output.

So running <exec command="phpunit ... "> [...]

and use --coverage-html and publish that in hudson and use --coverage-clover and the junit switch to get the coverage information into hudson. ( See jenkins-php.org or Setting up jenkins for php projects (there is a demo phpunit.xml.dist) for reference)

But chances are you don't want to so that :/

Saidly from the Phing docs of the phpunit task i see no way of telling phing directly to use --process-isolation switch.

So maybe someone else has a solution for that. I don't have one that will still produce code coverage.

From the comments

I'm going to assume that you have 2 separate pieces of test code that can't be run in a single process so what would ideal is if you could say:

run 2 processes: "testsuite one do that, testsuite two do the other thing"
after that aggregate the results

Sadly i don't know how you could tell phpunit to do so. I'll investigate further but for now the only thing i know that works for sure is to use --process-isolation each test in the whole testsuite. If there is a way to run a whole suite in one separate process i don't know about it.

Hopefully someone else has an easier solution to this :)

It seems this is simply the way Phing works. It's not clear from the Phing docs, but I've found a few cases in the mailing list where users are asking similar questions, and the answers tend to be to rename one of the offending classes... which clearly isn't much use when you're using them as mocks. (eg http://phing.tigris.org/ds/viewMessage.do?dsForumId=1083&dsMessageId=2426985&orderBy=createDate&orderType=desc, http://phing.tigris.org/ds/viewMessage.do?dsForumId=1083&dsMessageId=2624737). In the absence of anything better, I'll accept your answer. It doesn't quite feel right tho. — Spudley, Mar 21 '11 at 11:38
Yeah, if i find out about a better way to do this I'll come back and edit — edorian, Mar 21 '11 at 11:40

multiple tests clashing in phing

1 Answers1

From the comments