How to parametrize Maven surefire plugin so I can choose which TestNG suites to run

Question

I've got many test suites in TestNG. These are XML files. I want to be able to choose multiple XML suites when running integration-test from maven.

Currently I can add the suite files to pom.xml like this:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <suiteXmlFiles>
      <suiteXmlFile>${pathToMySuiteFile_1}</suiteXmlFile>
      <suiteXmlFile>${pathToMySuiteFile_1}</suiteXmlFile>
    </suiteXmlFiles>
  </configuration>
</plugin>

This solution has some limitations. I can only change a path to the test suite I've got defined in pom.xml. So in my example it always has to be two files. I'm not able to run, lets say, 5 suites or just one.

Is there a way to somehow parametrize the whole section "suiteXmlFiles" in pom.xml ?

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <suiteXmlFiles>
      ${multiple_paths_ToMySuiteFiles}
    </suiteXmlFiles>
  </configuration>
</plugin>

Running everything that matches given test group is not an option for me: I don't want to load all the suites I've got and then run just the selected tests using groups in TestNG suite. The reason being that a report that gets generated after running all the test suites with group filters is different from a report when just the selected test suites were run.

score 22 · Answer 1 · answered Jul 15 '15 at 21:36

22

According to User Property of suiteXmlFiles You can use:

mvn test -Dsurefire.suiteXmlFiles=test1.xml,test2.xml

answered Jul 15 '15 at 21:36

Sam R.

16,027
12
69
122

Using this solution, you don't need to specify the tag in the pom.xml file. Best answer. – Tyler H Jul 29 '16 at 21:49
with this one I get error `[ERROR] Unknown lifecycle phase ".suiteXmlFiles=test1.xml,test2.xml". You must specify a valid lifecycle phase or a goal in the format : or :[:]:.` – GetoX Sep 10 '20 at 09:50
1

No doubt it is the best answer, I was struggling for some days with same issue. With this, it is resolved now. Thanks @Sam – Saurabh Nov 16 '22 at 07:11

mekdev · Answer 2 · 2012-12-11T23:23:44.687

We also hit this issue with our tests. The current workaround that we use now is to define a property variable in the property section and inject it in sure-fire suiteXmlFiles block.

<properties>
    <!-- Default suites -->
    <batsSuiteFile>${project.build.testOutputDirectory}/BatsTests.xml</batsSuiteFile>
    <smokeSuiteFile>${project.build.testOutputDirectory}/SmokeTests.xml</smokeSuiteFile>

    <!-- Default suite files if not being specified from mvn command line -->
    <defaultSuiteFiles>${batsSuiteFile},${smokeSuiteFile}</defaultSuiteFiles>
    <suiteFile>${defaultSuiteFiles}</suiteFile>
</properties>

Then in the plugin section...

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.12</version>
    <configuration>
       <suiteXmlFiles>
          <!-- Suite file injection parameter from command line -->
          <suiteXmlFile>${suiteFile}</suiteXmlFile>
       </suiteXmlFiles>
    </configuration>
</plugin>

If nothing is specified from the command line it will fall back and default to the 2 suites specified above in the properties section. Then if you want to kick off a specified set of suite files you can do:

mvn test -DsuiteFile=test1.xml,test2.xml

Surprisingly from the maven docs you expect that the arg suiteXmlFiles should just override this from the command line and should just accept a comma delimited list of testng xmls http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html#suiteXmlFiles

If anyone has any other better way please share.

other way is to run below set of commands one after the other mvn -D tests ${path}/test1 test , mvn -D tests ${path}/test2 test and dependency is org.apache.maven.plugins maven-surefire-plugin 2.12 ${tests}.xml — Badrinath, Jul 15 '14 at 06:53
Use `-Dsurefire.suiteXmlFiles` instead. See my answer below. — Sam R., Jul 15 '15 at 21:38

score 0 · Answer 3 · answered Jul 09 '12 at 15:03

0

You can use groups parameter or specify -Dgroups=... in the command line:

(TestNG/JUnit47 provider with JUnit4.8+ only) Groups for this test. Only classes/methods/etc decorated with one of the groups specified here will be included in test run, if specified. For JUnit, this parameter forces the use of the 4.7 provider This parameter is ignored if the suiteXmlFiles parameter is specified. .

answered Jul 09 '12 at 15:03

Eugene Kuleshov

31,461
5
66
67

I mentioned it that I don't want to use groups as it breaks test result report and is not flexible enough anyway – maestr0 Jul 09 '12 at 16:53
Flexible or not, but that's your only choice. It can be as flexible as granular your groups are. Of course besides running individual tests. – Eugene Kuleshov Jul 10 '12 at 13:18

How to parametrize Maven surefire plugin so I can choose which TestNG suites to run

3 Answers3

Linked