1

I'm running tests with parallel execution using surefire, and that all seems to work fine.

However, there's only one testng-results.xml generated in the target/surefire-reports/ folder which ... only contains the results of the test that ran last.

I found an issue exactly for that reported for an older version of surefire, and it says "won't fix" here:

However, I doubt that I'm the first person on the planet who is trying to run unit tests in parallel with Jenkins and wants the results displayed properly using the TestNG Jenkins plugin, so I'm pretty sure there must be a solution for this, right?

Here's my surefire plugin config:

    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>${basepom.plugin.surefire.version}</version>
      <configuration>
        <forkCount>2.5C</forkCount>
        <reuseForks>false</reuseForks>
      </configuration>
    </plugin>

Here's the link to the surefire config btw.:

mac
  • 2,672
  • 4
  • 31
  • 43

1 Answers1

0

I actually ended up finding a solution that worked for me.

I think the problem is trying to do it with forkCount / reuseForks, so I've set those back to the default (you could also just skip those properties if you're not trying to overwrite something from a base pom etc.).

Instead I've used parallel and threadCount. Those however only apply to TestNG, but then, I also need it for TestNG (not JUnit).

This makes tests run in parallel, but the testng-results.xml is generated correctly (without being overwritten by each test running in parallel).

More details here:

Here's the plugin configuration I have now:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <!-- those are the two default values, you can probably skip them -->
    <forkCount>1</forkCount>
    <reuseForks>true</reuseForks>
    <!-- that's what made it work -->
    <parallel>classes</parallel>
    <threadCount>10</threadCount>
  </configuration>
</plugin>

Obviously, the thread count could be lower or higher depending on what you want to do / what specs your server has, and you could change the settings depending on whether you want to run classes, or some other "level" in parallel.

Also, you could configure parallel and threadCount together with any other properties in the suite file, if you choose to use a suite file.

mac
  • 2,672
  • 4
  • 31
  • 43
  • Related but for users using JUnit - I was porting a TestNG based suite to JUnit and was sad to discover that using `parallel` in `failsafe` with JUnit delays anything going to stdout until the all tests have completed. – markdsievers Mar 01 '17 at 19:18
  • Did you run into a problem working with Log4j in one of your classes under tests? I've been struggling in a problem where my class under test creates an static final instance of the Logger class, but when I run "mvn clean verify" using the classes and any number bigger than 0 I'm getting errors like Unexpected method call RepositorySelector.getLoggerRepository(): RepositorySelector.getLoggerRepository(): expected: 2, actual: 3 which is being called by Logger.getLog(anyClass); – victor.herrera Mar 24 '17 at 17:28