How to prevent maven surefire plugin from running members of a test suite twice?

Question

Given the following JUnit tests, surefire will run TestA and TestB twice. The first time a individual tests, and the second time as part of the test suite. It seems an unwanted behaviour. In fact popular IDE such as IntelliJ will just run the test once as part of the test-suite.

Is this a bug? Is there a way to configure surefire-plugin to run these thest only as part of a test-suite?

Sample Project available on GitHub

@RunWith(Suite.class)
@Suite.SuiteClasses({TestA.class, TestB.class})
public class TestSuite
{}

public class TestA {
    @Test
    public void test() {
        System.out.println("=== RUN " + this.getClass());
    }
}

public class TestB {
    @Test
    public void test() {
        System.out.println("=== RUN " + this.getClass());
    }
}

You could do this with juni4 `@Category`. Take a look to [this question](http://stackoverflow.com/questions/14132174/how-to-exclude-all-junit4-tests-with-a-given-category-using-maven-surefire). Hope it helps — troig, Sep 26 '14 at 12:35
@troig I tried that but doesn't work. Excluding TestA and TestB by marking them with a `@Category` will exclude them from the suite too. See the branch [category-test](https://github.com/spaceCamel/surefire-test-suite-execution/tree/category-test) for test case. — acorello, Sep 26 '14 at 13:13
In fact I was thinking in doing on the contrary (excluding the Suite marking with a `@Category` and execute the tests individually). It's not a valid way for you? — troig, Sep 26 '14 at 14:20
Test suites are part of JUnit. What I'm trying to find out is if/how surefire supports them. For the reasons to use test-suite I'd redirect to docs available online. In my case I intend to perform an expensive setup that will be shared by all the members of the test suite. — acorello, Sep 26 '14 at 15:35

score 5 · Answer 1 · answered Sep 26 '14 at 16:23

Surefire runs only classes whose name starts or ends in "Test".

But if a class is explicitly listed in a test suite it will be executed regardless of the name pattern.

So a solution could be to strip "Test" from the class names that are part of the suite.

Example

@RunWith(Suite.class) @Suite.SuiteClasses({A.class, B.class})
public class TestSuite
{}

public class A {
    @Test
    public void test() {
        System.out.println("=== RUN " + this.getClass());
    }
}

public class B {
    @Test
    public void test() {
        System.out.println("=== RUN " + this.getClass());
    }
}

Runnable example on branch [solution-renaming-classes](https://github.com/spaceCamel/surefire-test-suite-execution/tree/solution-renaming-classes) — acorello, Sep 26 '14 at 16:26

score 2 · Answer 2 · answered Sep 26 '14 at 12:38

Is there a way to configure surefire-plugin to run these thest only as part of a test-suite?

Yes. You can specify which tests to run and only include the suite:

  <plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
      <includes>
        <include>TestSuite.class</include>
      </includes>
    </configuration>
  </plugin>

A and B will no longer be run individually. Note that if you write a C test now, it won't run at all unless you add it to the test suite.

How to prevent maven surefire plugin from running members of a test suite twice?

2 Answers2