0

The main problem is when I use VintageTestEngine I can't search and run selected tests that are part of the data-driven test method.

For example, my test method called name LoginPageTestSuite_ # parameters.TestCaseName is fed with data in the AkeneoLoginPageTestSuite class. This test method is fed with a list of test data and thus creates several test cases with different test data. When this test method starts, 4 test cases are created and each has a different name resulting from adding the # parameters.TestCaseName parameter to the main method name.

The VintageTestEngine has methods that allow you to search and run specific test methods. In my case, I would like to run not the whole set with 4 tests, but only one which at the time of execution will have the name LoginPageTestSuite_TC_003_VerifyingLogin_LoginAndPassword.

I tried to debug VintageTestEngine and if I understand it correctly in the MethodSelectorResolver class (package org.junit.vintage.engine.discovery) the method shouldRun() has a big relationship with my problem.

private static Filter matchMethodDescription (final Description desiredDescription) {
        return new Filter () {
            public boolean shouldRun (Description description) {
                if (! description.isTest ()) {
                    Iterator var2 = description.getChildren (). Iterator ();

                    Description each;
                    up to {
                        if (! var2.hasNext ()) {
                            return false;
                        }

                        each = (Description) var2.next ();
                    } while (! this.shouldRun (each));

                    return true;
                } else {
                    return desiredDescription.equals (description) || this.isParameterizedMethod (description);
                }
            }

This code will return false in my case because, desiredDescription.equals (description), desiredDescription - LoginPageTestSuite_TC_003_VerifyingLogin_LoginAndPassword, description - LoginPageTestSuite_ # parameters.TestCaseName

My code for Spock test class and VintageTestEngine class.

package TestSuites.LoginPageTestSuite

import Assertions.LoginPageAsserts
import DataSourceExtensions.TestDataSource
import Model.LoginPageData
import PageObjects.LoginPageObject
import TestSuites.BaseTest
import com.google.common.base.Strings
import org.junit.jupiter.api.DisplayName
import org.openqa.selenium.WebDriver
import org.openqa.selenium.chrome.ChromeDriver
import spock.lang.Shared
import spock.lang.Unroll

class AkeneoLoginPageTestSuite extends BaseTest{

    @TestDataSource(TestSourcePath = "TestSuites/TestCaseData/LoginPageTestCaseData.json")
    private @Shared List<LoginPageData> _testCaseData

    private LoginPageObject _loginPage

    @DisplayName(value = "LoginPageTestSuite_#parameters.TestCaseName")
    @Unroll
    def "LoginPageTestSuite_#parameters.TestCaseName"(LoginPageData parameters)
    {
        given:
        _loginPage = new LoginPageObject()

        when:
        _loginPage.FillUserNameAndPassword(parameters.UserName, parameters.Password)

        then:
        if(!Strings.isNullOrEmpty(parameters.InvalidLoginMessage))
            LoginPageAsserts.VerifyingLoginPageMessageBox(
                    parameters.InvalidLoginMessage, _loginPage.getIncorrectLoginMessageBoxText())

        where:
        parameters << _testCaseData
    }
package Runners

import Configurations.Configuration
import org.junit.platform.engine.EngineDiscoveryRequest
import org.junit.platform.engine.ExecutionRequest
import org.junit.platform.engine.TestDescriptor
import org.junit.platform.engine.UniqueId
import org.junit.platform.engine.discovery.MethodSelector
import org.junit.platform.launcher.core.LauncherDiscoveryRequestBuilder
import org.junit.vintage.engine.VintageTestEngine

class AkeneoRunner {

    static void main(String[] args){
        new Configuration()

        VintageTestEngine engine = new VintageTestEngine()
        EngineDiscoveryRequest discovery = LauncherDiscoveryRequestBuilder.request().selectors(
                new MethodSelector(Class.forName("TestSuites.LoginPageTestSuite.AkeneoLoginPageTestSuite"),
                        "LoginPageTestSuite__TC_003_VerifyingLogin_LoginAndPassword"))
                .build()
        UniqueId root = UniqueId.root("LoginPageTestSuite__TC_003_VerifyingLogin_LoginAndPassword", "LoginPageTestSuite__TC_003_VerifyingLogin_LoginAndPassword")

        TestDescriptor testDescriptor = engine.discover(discovery, root)

        engine.execute(new ExecutionRequest(testDescriptor, new NoOpEngineExecutionListener(), new NoConfigurationParameters()))
    }
}

Here is the pom.xml file I am using:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>org.akeneo.groovy</groupId>
  <artifactId>AkeneoGroovy</artifactId>
  <version>1.0-SNAPSHOT</version>

  <name>AkeneoGroovy</name>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.7</maven.compiler.source>
    <maven.compiler.target>1.7</maven.compiler.target>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.spockframework</groupId>
      <artifactId>spock-core</artifactId>
      <version>1.3-groovy-2.5</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>3.141.59</version>
    </dependency>
    <dependency>
      <groupId>org.yaml</groupId>
      <artifactId>snakeyaml</artifactId>
      <version>1.25</version>
    </dependency>
    <dependency>
      <groupId>com.microsoft.sqlserver</groupId>
      <artifactId>mssql-jdbc</artifactId>
      <version>6.1.0.jre8</version>
    </dependency>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-core</artifactId>
      <version>2.12.0</version>
    </dependency>
    <dependency>
      <groupId>commons-pool</groupId>
      <artifactId>commons-pool</artifactId>
      <version>1.6</version>
    </dependency>
    <dependency>
      <groupId>commons-io</groupId>
      <artifactId>commons-io</artifactId>
      <version>2.6</version>
    </dependency>
    <dependency>
      <groupId>commons-dbcp</groupId>
      <artifactId>commons-dbcp</artifactId>
      <version>1.4</version>
    </dependency>
    <dependency>
      <groupId>org.junit.platform</groupId>
      <artifactId>junit-platform-engine</artifactId>
      <version>1.7.0-M1</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter</artifactId>
      <version>RELEASE</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.platform</groupId>
      <artifactId>junit-platform-launcher</artifactId>
      <version>1.7.0-M1</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.vintage</groupId>
      <artifactId>junit-vintage-engine</artifactId>
      <version>5.7.0-M1</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <artifactId>maven-clean-plugin</artifactId>
          <version>3.1.0</version>
        </plugin>
        <plugin>
          <artifactId>maven-resources-plugin</artifactId>
          <version>3.0.2</version>
        </plugin>
        <plugin>
          <artifactId>maven-compiler-plugin</artifactId>
          <version>3.8.0</version>
        </plugin>
        <plugin>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.22.1</version>
        </plugin>
        <plugin>
          <artifactId>maven-jar-plugin</artifactId>
          <version>3.0.2</version>
        </plugin>
        <plugin>
          <artifactId>maven-install-plugin</artifactId>
          <version>2.5.2</version>
        </plugin>
        <plugin>
          <artifactId>maven-deploy-plugin</artifactId>
          <version>2.8.2</version>
        </plugin>
        <plugin>
          <artifactId>maven-site-plugin</artifactId>
          <version>3.7.1</version>
        </plugin>
        <plugin>
          <artifactId>maven-project-info-reports-plugin</artifactId>
          <version>3.0.0</version>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
</project>

Maybe its now more clear, if not please let me know. Thanks

  • Welcome to SO. Please read the article about [MCVE](https://stackoverflow.com/help/mcve) and try to edit your question in order to become one. That would significantly improve your chances of getting helpful answers. I do not even know which Spock and Groovy versions you use. Build configuration, package names, imports, etc. - there is a lot which can cause problems. Just imagine for a minute you were me, seeing only the information given here and trying to answer the question. Could you? – kriegaex May 19 '20 at 14:49
  • Hello kriegaex, thanks for message. I edited my post and I hope you will understand it well :) – Michał Cichoń May 19 '20 at 18:50
  • This looks quite complex, I do not have much experience with data providers for iterated tests. Please share a full MCVE on GitHub. Thank you. It also strikes me as strange that you run your tests from a class with a main method instead of just using the usual Maven or Gradle plugins for running tests. And why you want to use JUnit 5 for Spock 1.3 which is based on JUnit 4, only to then use a vintage test engine, eludes me too. If you want to run Spock tests on JUnit 4, just use Junit 4. If you want to run them on the Jupiter platform, switch to Spock 2.0-M2 which is based on Jupiter anyway. – kriegaex May 20 '20 at 08:01
  • Ok I will share this. I have mixed up the librarys a bit but this is because it is new technology for me and I don't quite understand sometimes ;) Thanks anyway :) – Michał Cichoń May 20 '20 at 08:36
  • For anyone interested in the subject https://github.com/spockframework/spock/issues/1167 – Michał Cichoń May 20 '20 at 14:12
  • See, that's exactly why I wanted an MCVE: I did not understand your explanation in plain prose. You do not wish to run a single test but a single **iteration** of a parametrised test. If I had understood your explanation, I could have told you right away that this is not possible and probably never will be. The whole point of having a parametrised test is to run multiple iterations, not single ones. For that you have non-parametrised tests. The granularity level for test runners just does not go that low. – kriegaex May 20 '20 at 15:30
  • Your and Björn Kautler help saved my time which I could devote to finding a solution. Unfortunately, it will be difficult to manage tests without it because if one of the 4 tests fails, you will need to turn off the whole set ... – Michał Cichoń May 20 '20 at 16:09
  • No, never deactivate failing tests, it is a bad practice (I call it an anti pattern). Hiding problems will not encourage you to solve them. Let them nag you and cause you pain whenever you see the red flag in the build. Fix the problem, do not hide it. There is no point in a "successful" build like this if there are known problems. Besides, if you do want to hide the problem without deactivating the whole parametrised feature method, who don't you go to the source of it and comment out a row in the data table or in the code of your dynamic data provider? – kriegaex May 21 '20 at 03:42
  • There are pros and cons, but I think if you have 50 fail tests and each takes a minute, you save time production code implementation 50 minutes. – Michał Cichoń May 22 '20 at 07:32
  • I do not know if you understand, 50 tests that fail are errors not in tests but in the application. What you write makes me realize that it makes no sense to continue this discussion. – Michał Cichoń May 22 '20 at 08:13
  • That's my whole point and even worse! If you have 50 known errors in the application, fix the application first (which also fixes the tests) before producing even more features on a crumbling foundation. And if you think you are too busy to fix them, at least make them transparent. I can always have green tests and a successful build if I `@Ignore` all failing ones. Then I don't need a test report though. It has no meaning anyway. – kriegaex May 22 '20 at 08:56
  • Sorry to be so assertive about this, but I am an agile coach and I have coached hundreds of developers in order to wrap their heads around this point and they were all much happier afterwards and produced better products by fixing problems before producing more features on top of a buggy product. Transparency, transparency, transparency! Then you can also better argue with your product owner and show her how dangerous it is if she expects even more features instead of investing time for cleaning up the existing mess first. You are just shooting your own foot. – kriegaex May 22 '20 at 09:00

0 Answers0