Is isolation of failsafe and surefire runs from each other using a skip approach possible?

Question

The value of property skipITs when overridden on the command line appears to be ignored by maven if used in conjunction with -Dit.test=full.Classname. The specified failsafe test will not be run (case one).

As opposed to this specifiing skipITs without the it.test switch leads to running of existing failsafe tests (case two).

Background: Am trying to isolate surefire tests from failsafe tests runs by namely either running both types of them or one of each only. The maven calls are:

mvn -Pintegration -DskipTests=true -DskipITs=false -Dit.test=full.Classname verify

in the first case and:

mvn -Pintegration -DskipTests=true -DskipITs=false verify

in the second.

The relevant configuration (pom.xml, snippets only) being:

<properties>
    <skipTests>false</skipTests>
    <skipITs>false</skipITs>
</properties>

(those are defaults) and

<profiles>
    <profile>
        <id>integration</id>
        <build>
            <plugins>
                <plugin>
                    <artifactId>maven-failsafe-plugin</artifactId>
                    <executions>
                        <execution>
                            <id>custom<id>
                            <goals>
                                <goal>integration-test</goal>
                                <goal>verify</goal>
                            </goals>
                            <configuration>
                                <skipTests>${skipITs}</skipTests>
                                <skip>${skipITs}</skip>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

Did you observe that too or have you eventually found a better working approach?

Why not using the defaults provided by the plugins. `-DskipITs=true` to skip integration tests of maven-failsafe-plugin and `-DskipTests=true` to skip the unit tests of maven-surefire-plugin. Why are you defining properties? — khmarbaise, Jul 17 '15 at 14:16
@khmarbaise `-DskipTests=true` will also prevent the Failsafe plugin to execute any tests. The OP wants to skip Surefire but run Failsafe. — hzpz, Jul 17 '15 at 15:46

score 2 · Answer 1 · answered Jul 17 '15 at 11:32

By default, the Surefire plugins runs during the test phase and usually you configure the Failsafe plugin to run during the integration-test and verify phase like this:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.18.1</version>
    <executions>
        <execution>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

To run the Surefire tests only, use

mvn clean test

To run both the Surefire and the Failsafe tests, use

mvn clean verify

You can completely skip any plugin by using the <skip> configuration option. If you configure the Surefire plugin like this:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.18.1</version>
    <configuration>
        <skip>${skip.surefire}</skip>
    </configuration>
</plugin>

you can run only the Failsafe tests by calling

mvn clean verify -Dskip.surefire=true

score 0 · Answer 2 · answered Jul 17 '15 at 11:38

If the integration test (it.test) has a non-default name I need to add an include pattern accordingly, like in (pom.xml snippet):

<profiles>
    <profile>
        <id>dbUnit</id>
        <build>
            <plugins>
                <plugin>
                    <artifactId>maven-failsafe-plugin</artifactId>
                    <executions>
                        <execution>
                            <id>custom</id>
                            <goals>
                                <goal>integration-test</goal>
                                <goal>verify</goal>
                            </goals>
                            <configuration>
                                <skipTests>${skipITs}</skipTests>
                                <skip>${skipITs}</skip>
                                <includes>
                                    <include>**/*DbTest.java</include>
                                </includes>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

But that appears valid only for the original approach. Also observe that the default inclusion patterns might need to be added too in case for your integration tests you followed different naming schemes.

You should use naming conventions which is already the default in maven-surefire/maven-failsafe instead of using includes etc. — khmarbaise, Jul 17 '15 at 14:17
Maven is all about convention over configuration. @khmarbaise is right, don't make your life hard by not following the conventions! Also, Stackoverflow does not work like a traditional discussion board. There is one question and everything else should just be answers to that question. Discussion can happen in the chat. — hzpz, Jul 17 '15 at 15:49

Is isolation of failsafe and surefire runs from each other using a skip approach possible?

2 Answers2