JUnit5: Surefire plugin runs JUnit4 tests twice

Question

I have both JUnit4 and JUnit5 tests in my project. The problem is that when I run mvn clean install the JUnit4 tests are run twice (JUnit5 tests run fine and once only).

I have the following surefire-plugin configuration (showing only relevant dependencies) in my parent project pom

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.22.2</version>
  <configuration>
    <threadCount>1</threadCount>
    <properties>
      <property>
        <name>junit</name>
        <value>false</value>
      </property>
    </properties>
  </configuration>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-testng</artifactId>
      <version>2.22.2</version>
    </dependency>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit-platform</artifactId>
      <version>2.22.2</version>
    </dependency>
  </dependencies>
</plugin>
...
...
<dependency>
  <groupId>org.junit.jupiter</groupId>
  <artifactId>junit-jupiter-engine</artifactId>
  <version>5.6.2</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.junit.vintage</groupId>
  <artifactId>junit-vintage-engine</artifactId>
  <version>5.6.2</version>
  <scope>test</scope>
</dependency>
<dependency>
  <!-- needed for https://youtrack.jetbrains.com/issue/IDEA-231927?_ga=2.101965186.223349104.1602977709-1646014256.1600106493 -->
  <groupId>org.junit.platform</groupId>
  <artifactId>junit-platform-launcher</artifactId>
  <version>1.6.2</version>
  <scope>test</scope>
</dependency>

I replicated the above surefire-plugin in the child project as well to make sure it does not get overridden by anything. But still, the JUnit4 tests are run twice.

Following is the surefire-plugin portion in the effective pom -

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.22.2</version>
  <configuration>
    <threadCount>1</threadCount>
    <properties>
      <property>
        <name>junit</name>
        <value>false</value>
      </property>
    </properties>
  </configuration>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-testng</artifactId>
      <version>2.22.2</version>
    </dependency>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit-platform</artifactId>
      <version>2.22.2</version>
    </dependency>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit47</artifactId>
      <version>2.19.1</version>
      <scope>compile</scope>
    </dependency>
  </dependencies>
</plugin

On doing some debugging with -X option I think the reason is because surefire-junit47 also gets added in the providers for surefire-plugin, surefire-junit-platform runs the junit4 tests once and they are run again by surefire-junit47 provider. If that is a probable reason then how can I prevent this from getting added to surefire-plugin dependencies? I tried <classpathDependencyExcludes> but that didn't help and the effective pom still contained surefire-junit47.

Also is there any way to avoid JUnit4 running twice even with having both providers (surefire-junit47 and surefire-junit-platform)?

---------- Update ------------

I also have set the junit property to false in the configuration for surefire to prevent testng provider running the junit tests (as suggested here). But still, I am getting two runs of JUnit4 tests. My guess is that somehow surefire-junit47(which is getting added mysteriously) and surefire-junit-platform are acting weird together to cause duplicated runs.

---------- Update ------------

The actual problem turns out that my project is inheriting a parent pom which has declared a dependency surefire-junit47 in surefire-plugin. So effectively my project has both surefire-junit-platform and surefire-junit47 which results in dual runs of JUnit4 tests.

Answer 1

I was able to reproduce your problem. You are running into a situation described here:

TestNG 6.5.1 and higher provides support to run TestNG and JUnit 4.x in current Maven project. (...)

You may want to run two providers, e.g. surefire-junit47 and surefire-testng, and avoid running JUnit tests within surefire-testng provider by setting property junit=false.

So please change your Surefire plugin configuration to:

<configuration>
  <threadCount>1</threadCount>
  <properties>
    <!-- Avoid running JUnit 4 tests in TestNG engine -->
    <property>
      <name>junit</name>
      <value>false</value>
    </property>
  </properties>
</configuration>

---

Update: How to (pseudo) exclude a direct Maven plugin dependency:

Okay, FWIW I came up with a hacky way to "exclude" a dependency by overriding it with a dummy.

First you add a module to your project's root POM, but without specifying the root POM as its parent. Just make sure the module is built in the same reactor. Alternatively, you can also create a separate project for it and make sure its resulting artifact is in your company's repository.

The module POM looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>org.apache.maven.surefire</groupId>
  <artifactId>surefire-junit47</artifactId>
  <version>dummy</version>
</project>

In your child POM where you wish to deactivate the JUnit 4.7 engine in favour of running your JUnit 4 tests on the JUnit 5 platform, you can do something like this:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>${maven-surefire.version}</version>
  <configuration>
    <threadCount>1</threadCount>
    <properties>
      <!-- Avoid running JUnit 4 tests in TestNG engine -->
      <property>
        <name>junit</name>
        <value>false</value>
      </property>
    </properties>
  </configuration>
  <dependencies>
    <dependency>
      <!-- Deactivate JUnit 4.7 engine by overriding it with an empty dummy -->
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit47</artifactId>
      <version>dummy</version>
    </dependency>
  </dependencies>
</plugin>

<!-- (...) -->

<dependencies>
  <dependency>
    <groupId>org.apache.maven.surefire</groupId>
    <artifactId>surefire-junit47</artifactId>
    <version>dummy</version>
  </dependency>
</dependencies>

This is super ugly and I would still recommend refactoring the parent POM, if possible. Anyway, it works. Now your JUnit 4 tests only run once, i.e. inside the JUnit 5 platform alongside JUnit 5 tests, Spock 2.0 tests or what have you.

---

Update 2: After you provided an MCVE on GitHub, I sent you a pull request implementing exactly the workaround I mentioned in the previous update. The important commit is this one.