How to conditionalize GUI tests using Netbeans/Maven vs maven on command line invocation

Question

I'd like to have a single project pom but have my GUI tests always run when I'm invoking JUnit on Netbeans, but have them conditional (on an environment variable?) when building on the command line (usually for production build - on a headless machine, but sometimes just for build speed).

I don't mind instrumenting my JUnit tests for this, as I already have to set up my GUI test infrastructure, but how do I conditionalize my pom!

Netbeans 6.5 with Maven plugin.

Any ideas how I can accomplish this?

Ilane

Answer 1

The Surefire plugin can specify which tests to run.

You could specify different sets to run for different profiles, and select your profile using -P{profilename}.

Answer 2

One solution would be to use a profile (that could be indeed activated by an environment variable) and to exclude GUI tests from the Maven Surefire Plugin configuration using excludes in this profile, based on a naming convention. Something like this:

<profile>
  <id>headless</id>
  <activation>
    <property>
      <name>env.FOO</name>
      <value>foo</value>
    </property>
  </activation>
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.5</version>
        <configuration>
          <excludes>
            <exclude>**/*$*</exclude><!-- to exclude inner classes -->
            <exclude>**/gui/**</exclude>
          </excludes>
        </configuration>
      </plugin>
      ...
    </plugins>
  </build>
</profile>

Another option would be to use TestNG and to leverage the excludeGroups parameter.

Answer 3

To achieve the desired behavior in my Maven project on Netbeans, I set up a profile defining an environment variable in my project's pom and modified the Test file action in my project's properties to activate my new profile. In this way I could check the enviroment variable in my tests. (This could have been done similarly with a system property.)

To keep from having to add a check to each and every GUI test, however, I found that I could add a JUnit filter, which would automatically ignore the UI tests in the case where I didn't want to run them and put the number of skipped tests in the test result line.

Here what my profile looks like:

      <profile>
          <id>test-gui</id>
          <build>
              <plugins>
                  <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                      <skipTests>false</skipTests>
                      <excludes>
                        <exclude>**/TestSuite.java</exclude>
                      </excludes>
                       <environmentVariables>
                        <GUI_TEST>true</GUI_TEST>
                      </environmentVariables>
                    </configuration>
                  </plugin>
                </plugins>
              </build>
      </profile>

Here's a description of how to update the Netbeans Action:

(Right click on project in Projects window)->Properies
    In Categories box, select 'Actions'
        In Actions box, select 'Test file'
            In Activate Profiles textfield, enter 'test-gui'
    Click 'OK' button to save.

To add the JUnit filter (and the documentation was sparse, so I may not have done this in the most effective manner), I subclassed the default test runner, TestClassRunner and had it call my Filter, which checks the test name and the environment variable. To get JUnit to call my class, I annotated the test class.

public class GUITestClassRunner extends TestClassRunner {

    public GUITestClassRunner(Class klazz) throws InitializationError {
        super(klazz);
    }

    public GUITestClassRunner(Class klazz, Runner runner) 
        throws InitializationError, NoTestsRemainException {
        super(klazz, runner);
    }

    @Override
    public void  run(RunNotifier notifier) {
        if (getDescription().testCount() > 0) {
            try {
                filter(new GUIFilter(notifier));
            } catch (NoTestsRemainException ex) {
                Description description = getDescription();
                notifier.fireTestStarted(description);
                notifier.fireTestIgnored(description);
                notifier.fireTestFinished(description);
                return;
            }
        }
        super.run(notifier);
    }

    class GUIFilter extends Filter {
        private boolean isGUI = false;
        private RunNotifier notifier;

        public GUIFilter(RunNotifier notifier) {
            this.notifier = notifier;
            isGUI = UI.isGUITestEnvironment();
        }

        @Override
        public boolean shouldRun(Description desc) {
            if (!isGUI && UI.isGUITest(desc.getDisplayName())) {
                Description description = getDescription();
                notifier.fireTestStarted(description);
                notifier.fireTestIgnored(description);
                notifier.fireTestFinished(description);
                return false;
            }
            return true;
        }

        @Override
        public String describe() {
            return "all tests except GUI tests if headless";
        }
    }
}

To call this runner, the test class needs to be annotated with:

@RunWith(GUITestClassRunner.class)
public class MyJUnitTestClass
{
    ....
    @Test
    public void testAlpha() {...}

    @Test
    public void testBeta_UI() {...}
}

So, now, from Netbeans, I just run my unit test class and the GUI tests automatically run. I can run from the command line with no environment variables set or GUI_TEST set to false, and my GUI tests are skipped, or I can run from the command line with GUI_TEST set to true, or use mvn -Ptest-gui and all my GUI tests run.

My sincerest thanks to Brian and Pascal for pointing me in the right direction.

Ilane