5

For a continuous integration build, we use JcCoCo to minimise the number of tests to be run. However, on some commits it determines that there are no tests worth running at all. For example, if only an image was changed.

Here is a snippet from the build.xml:

<fileset id="int.tests" dir="${build.inttest.source}/java">
  <include name="**/*Test.java"/>
</fileset>

<taskdef name="testng" classname="org.testng.TestNGAntTask"
  classpath="${build.jars.test}/testng.jar"/>

<jacoco:coverage destfile="./inttest-jacoco.exec">
    <testng outputDir="./reports/intTest" failureproperty="testNGFailed" haltonfailure="false"
      verbose="2" workingDir="${build.dir}" classfilesetref="int.tests">
      <classpath>
        <path refid="build.inttest.classpath"/>
      </classpath>
    </testng>
</jacoco:coverage>

When no tests are run, property testNGFailed is set to true and the build subsequently fails.

The logging in this scenario looks like this:

13:16:49,116 INFO  -    [testng] ===============================================
13:16:49,116 INFO  -    [testng] Ant suite
13:16:49,116 INFO  -    [testng] Total tests run: 0, Failures: 0, Skips: 0
13:16:49,116 INFO  -    [testng] ===============================================
13:16:49,116 INFO  - 
13:16:49,191 WARN  -    [testng] [TestNG] No tests found. Nothing was run

How can I make the build pass when there are no test to be run, but fail when any test fails?

Can I get Jacoco to always run at least one test?

Can I get TestNG to only set the failureproperty when a test has failed?

WW.
  • 23,793
  • 13
  • 94
  • 121

1 Answers1

0

First of all - JaCoCo does not execute your tests! Tests are executed by TestNG or JUnit or whatever testing framework - JaCoCo just collects coverage information from this execution.

According to http://testng.org/doc/ant.html :

classfilesetref - A reference to a ResourceCollection containing the test classes to be run

whereas in your case it points on directory containing .java files, so seems logical that TestNG can't find tests.

Godin
  • 9,801
  • 2
  • 39
  • 76
  • I mustn't be asking the question right. It usually finds tests and runs them fine. The build fails when there are no tests found that need to be run, based on the change committed and the previous coverage informtation collected. – WW. Mar 22 '18 at 15:35