1

How can I continue running a feature on failure of 1 scenario to next?

Currently if two Scenarios fails out of 10 (lets say) next 8 will not execute.

I am using Cucumber with Java, Junit, Maven on Windows 10.

Edit: Here is my pom, hope it helps: Could you tell me which of dependencies should I update in your oppinion?

    <properties>
        <java.version>11</java.version>
        <cucumber.version>6.8.0</cucumber.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>org.hibernate</groupId>
                    <artifactId>hibernate-entitymanager</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.hibernate</groupId>
                    <artifactId>hibernate-core</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-core</artifactId>
            <version>5.2.10.Final</version>
        </dependency>
        <dependency>
            <groupId>javax.xml.bind</groupId>
            <artifactId>jaxb-api</artifactId>
            <version>2.3.0</version>
        </dependency>
        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <version>42.2.11</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>${httpclient.version}</version>
            <exclusions>
                <exclusion>
                    <artifactId>commons-logging</artifactId>
                    <groupId>commons-logging</groupId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpcore</artifactId>
            <version>${httpcore.version}</version>
        </dependency>
        <dependency>
            <groupId>com.google.code.gson</groupId>
            <artifactId>gson</artifactId>
            <version>2.8.6</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!-- marshalling -->
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.20</version>
            <scope>provided</scope>
        </dependency>

        <!-- Test deps -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>3.141.59</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>htmlunit-driver</artifactId>
            <version>2.51.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-core</artifactId>
            <version>${cucumber.version}</version>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>${cucumber.version}</version>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-spring</artifactId>
            <version>${cucumber.version}</version>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>${cucumber.version}</version>
        </dependency>
        <dependency>
            <groupId>org.junit.vintage</groupId>
            <artifactId>junit-vintage-engine</artifactId>
            <version>5.7.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-redis</artifactId>
        </dependency>
        <dependency>
            <groupId>redis.clients</groupId>
            <artifactId>jedis</artifactId>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.11</version>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>RELEASE</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>javax.mail</groupId>
            <artifactId>mail</artifactId>
            <version>1.4.7</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
                <configuration>
                    <argLine>-Xmx2048m</argLine>
                    <includes>
                        <include>CucumberE2ETest.java</include>
                    </includes>
                    <testFailureIgnore>true</testFailureIgnore>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

1 Answers1

0

Are you writing individual tests for each scenario? If you currently have everything in one big block in a single method, you could break it up into fine-grain, detailed tests, with each test addressing one specific aspect. This way, if something breaks, you can see exactly where it breaks and fix it right away! Additionally, you can run all tests in one go, and you will be able to see which tests fail and for what reason. If that isn't the case, please provide more info :D

EDIT: The pom you have provided has multiple dependencies that might be clashing:

<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-junit</artifactId>
    <version>${cucumber.version}</version>
</dependency>
<dependency>
    <groupId>org.junit.vintage</groupId>
    <artifactId>junit-vintage-engine</artifactId>
    <version>5.7.0</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>RELEASE</version>
    <scope>test</scope>
</dependency>

Could you please provide the error message that results in running the tests? Maybe there could be some useful information that can help guide us further to the root cause.

Please also check out this answer: https://stackoverflow.com/a/63845288/17105581 Here the author of the answer explains that cucumber-junit-platform-engine might be needed.

Let me know if this helps! :D

ferrouskid
  • 466
  • 2
  • 7
  • Currently I have around 15 methods for 50 scenarios, but I think this is OK, because for most of the time I am changing just the testing data. I found simular question here - https://stackoverflow.com/questions/54691114/if-one-scenario-fails-continue-with-next-scenario-in-a-cucumber-feature-file and this user resolved it by upgrading the surefire plugin to latest version. The thing is that my surefire plugin is using the latest version right now. – user1167642 Dec 09 '21 at 16:50
  • Interesting. You are saying that you're changing the test data - are the later tests dependant on the outcome of the previous tests? – ferrouskid Dec 09 '21 at 17:08
  • Another thing, it could be different dependencies that are messing up - I ran into this before unfortunately, updating one seemingly unrelated dependency fixed the issue. – ferrouskid Dec 09 '21 at 17:09
  • Tests steps of course are dependent, but different scenarios not, they do not rely on the outcome of the previous one, if I am not wrong of course. – user1167642 Dec 09 '21 at 17:23
  • 1
    ferrouskid do you need some more information? – user1167642 Dec 09 '21 at 20:57
  • 1
    The most important information that seems to be missing is how to reproduce your problem. So you have to remove everything that is not essential to reproducing the problem. – M.P. Korstanje Dec 09 '21 at 22:08
  • I agree with MP Korstanje. If you could maybe provide a snippet of the test code or maybe show the output of why it fails, that could give some insight into the issue. I'd be happy to have a look and see if I can reproduce the error :) – ferrouskid Dec 10 '21 at 09:59
  • Have you had a look at the edit I provided and the link to the other stackoverflow answer? Let me know if it helps :D – ferrouskid Dec 12 '21 at 12:55
  • Not really. Too many moving parts. Cucumber, JUnit 4, JUnit Vintage, JUnit 5, Surefire, Spring, ecy. Can you make a small repo on GitHub that reproduces the problem with out all the extra stuff. – M.P. Korstanje Dec 12 '21 at 15:06