Maven site reporting fine for surefire tests, needs extra config for pitest mutationCoverage

Question

If I set up the <reporting> section in my pom as follows, I only get the surefire report, while the pitest report fails because it can't find any input.

  <reporting>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-project-info-reports-plugin</artifactId>
        <version>2.9</version>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-report-plugin</artifactId>
        <version>2.19.1</version>
      </plugin>
      <plugin>
        <groupId>org.pitest</groupId>
        <artifactId>pitest-maven</artifactId>
        <version>1.1.10</version>
        <configuration>
          <targetClasses>
            <param>pricingengine.*</param>
          </targetClasses>
          <targetTests>
            <param>pricingengine.*</param>
          </targetTests>
        </configuration>
        <reportSets>
          <reportSet>
            <reports>
              <report>report</report>
            </reports>
          </reportSet>
        </reportSets>
      </plugin>
    </plugins>
  </reporting>

To obtain the input to the pitest report so that it outputs to the site reports, I need to do this first:

mvn compile test-compile org.pitest:pitest-maven:mutationCoverage

Do I have to set up each of these in the <build> section as plugins with executions bound to the pre-site phase to get this to happen? Or is there a simpler solution with another plugin I'm not aware of?

@GeraldMücke Yes exactly. It might be run on a CI server slave where there is nothing else, a totally blank environment. — Adam, Jun 16 '16 at 10:19

Gerald Mücke · Accepted Answer · 2016-06-16T06:12:51.050

1

The maven-surefire-report-plugin however explicity states, that it invokes the test goal of the default lifecycle. The pitest plugin doesn't. So yes, you have to add the pitest-maven plugin to the build section and bind it to a lifecycle phase, i.e. pre-site. I wouldn't recommend the use of the site lifecycle for that purpose as it is not intended for long-running analysis tasks, but that is up to you.

So the build order is:

build lifecycle
- build the module (compile phase)
- run the test (test phase)
- run the mutation coverage (i.e. in verify phase)
site lifecycle
- pre-site (mutationCoverage);
- generate reports
- publish reports
- ...

I'd suggest to use a profile so the mutation test is not run on every build and you can activate it when needed (i.e. mvn site-P pit)

<profile>
  <id>pit</id>
  <build>
    <plugins>
        <plugin>
            <groupId>org.pitest</groupId>
            <artifactId>pitest-maven</artifactId>
            <configuration>
                <targetClasses>
                    <param>pricingengine.*</param>
                </targetClasses>
                <targetTests>
                    <param>pricingengine.*</param>
                </targetTests>
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>mutationCoverage</goal>
                    </goals>
                    <phase>pre-site</phase>
                </execution>
            </executions>
        </plugin>
    </plugins>
  </build>
</profile>

edited Jun 16 '16 at 06:12

answered Jun 15 '16 at 09:25

Gerald Mücke

10,724
2
50
67

I've actually bound it to the pre-site phase because the site lifecycle is the one I really need it for. Ideally I'd like to tie in the site lifecycle with the deploy / release lifecycle, but that's another story. – Adam Jun 15 '16 at 20:50
But that doesn't make sense, "the site lifecycle handles the creation of your project's site documentation", not for actually doing analysis tasks. Of course you could do it that way, but you shouldn't. Mutation testing is not about generating reports, it's about finding gaps in your test suite. – Gerald Mücke Jun 15 '16 at 20:55
the fact is that the pitest-maven reporting plugin needs that test output, because it doesn't generate it itself. The surefire reporting plugin however does invoke the compile, test-compile and surefire tests. I need to mimic that for the pitest plugin. – Adam Jun 15 '16 at 21:46
in that case, just bind the execution to `pre-site` (I updated response accordingly) – Gerald Mücke Jun 16 '16 at 06:13
can I bind to the lifecycle too? If I bind `pitest-maven` to the `pre-site` phase it fails because it needs the compiled tests. So then I bind the `compiler:test-compile` goal to the `pre-site` phase too, but then that fails because it needs the compiled source code. So that means I have to bind `compiler:compile` to `pre-site` as well. How do I bind the `test lifecycle` instead, so that at least the source is compiled automatically for the tests? – Adam Jun 16 '16 at 16:10
1

the surefire-report-plugin defines it's own lifecycle that defines the test-phase. Custom maven plugins usually don't define a custom lifecycle. So if you'd like to resemble the build lifecycle you have to define all goals necessary in the pom which could be a tedious task. Maybe it's possible to run the report plugin before mutationCoverage. But again, I wouldn't do it that way. It's probably more easy to just run the build lifecycle before generating the report, like `mvn install site`. If the code is already compiled, it won't take that long (especially in comparison to mutationCoverage). – Gerald Mücke Jun 16 '16 at 17:41

Maven site reporting fine for surefire tests, needs extra config for pitest mutationCoverage

1 Answers1