Why is throw new SkipException() skipping all my methods?

Question

Why is throw new SkipException() skipping all my methods across all classes?

Class one has a method which fails therefore the remaining methods in that class are skipped and the skipcounter is set to 0.
Class two should execute all methods and all methods should pass in class two because the skip counter was set to 0 and after class one was executed.

The following code seems to skip all methods even though there is only one assert.fail which is contained in class one?

Test Base:

import org.testng.annotations.BeforeMethod;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeClass;
import org.testng.ITestResult;
import org.testng.SkipException;

  public class TestBase {
        static int numb = 0;
      @BeforeMethod
      public void beforeMethod() {
          if(numb == 1) {
                throw new SkipException("Skipping this test");
          }  
      }

      @AfterMethod
      public void afterMethod(ITestResult testResult) {
            if (testResult.getStatus() == ITestResult.FAILURE) {
                numb = 1;
            }
      }

      @BeforeClass
      public void beforeClass() {
          TestBase.numb = 0;
      }

Test1:

import org.testng.annotations.Test;

public class Test1 extends TestBase {
    @Test(priority = 1)
    public void test1() throws Exception {
        Assert.fail();
    }

    @Test(priority = 2)
    public void test2() throws Exception {
        System.out.println("method2");
    }

    @Test(priority = 3)
    public void test3() throws Exception {
        System.out.println("method3");
    }
}

Test2:

import org.testng.annotations.Test;

public class Test2 extends TestBase {
    @Test(priority = 1)
    public void test1() throws Exception {
        System.out.println("method4");
    }

    @Test(priority = 2)
    public void test2() throws Exception {
        System.out.println("method5");
    }

    @Test(priority = 3)
    public void test3() throws Exception {
        System.out.println("method6");
    }
}

Console output:

..
... TestNG 6.9.10 by Cédric Beust (cedric@beust.com)
...

[TestNG] Running:
  C:\Users\Gianni.Bruno\Desktop\BuyAGiftFramework\BuyAGiftFramework\Test.xml

[TestRunner] Starting executor for test BuyAGift Automation Tests with time out:2147483647 milliseconds.
SKIPPED CONFIGURATION: @BeforeMethod beforeMethod
SKIPPED CONFIGURATION: @AfterMethod afterMethod
SKIPPED CONFIGURATION: @BeforeMethod beforeMethod
SKIPPED CONFIGURATION: @AfterMethod afterMethod
SKIPPED CONFIGURATION: @BeforeClass beforeClass
SKIPPED CONFIGURATION: @BeforeMethod beforeMethod
SKIPPED CONFIGURATION: @AfterMethod afterMethod
SKIPPED CONFIGURATION: @BeforeMethod beforeMethod
SKIPPED CONFIGURATION: @AfterMethod afterMethod
SKIPPED CONFIGURATION: @BeforeMethod beforeMethod
SKIPPED CONFIGURATION: @AfterMethod afterMethod
FAILED: test1
junit.framework.AssertionFailedError
    at junit.framework.Assert.fail(Assert.java:47)
    at junit.framework.Assert.fail(Assert.java:53)
    at BuyAGiftFramework.test.Test1.test1(Test1.java:9)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:86)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:643)
    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:820)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1128)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

SKIPPED: test2
SKIPPED: test3
SKIPPED: test1
SKIPPED: test2
SKIPPED: test3

===============================================
    BuyAGift Automation Tests
    Tests run: 6, Failures: 1, Skips: 5
    Configuration Failures: 0, Skips: 11
===============================================


===============================================
BuyAGift_Automation_Scripts_by_GBruno
Total tests run: 6, Failures: 1, Skips: 5
Configuration Failures: 0, Skips: 11
===============================================

[TestNG] Time taken by org.testng.reporters.JUnitReportReporter@366e2eef: 14 ms
[TestNG] Time taken by org.testng.reporters.XMLReporter@1936f0f5: 16 ms
[TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@3f2a3a5: 32 ms
[TestNG] Time taken by org.testng.reporters.jq.Main@614c5515: 94 ms
[TestNG] Time taken by [FailedReporter passed=0 failed=0 skipped=0]: 16 ms
[TestNG] Time taken by org.testng.reporters.EmailableReporter2@6537cf78: 19 ms

Test Execution XML:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Automation_Scripts_by_GB" verbose="2" parallel="classes" thread-count="1">

    <test name="BuyAGift Automation Tests">
        <packages>
            <package name="BuyAGiftFramework.test.*"></package>
        </packages>
    </test>
</suite>

Because variable 'numb' is static and its value will remain across all instances... If tests in Test2 ran before those in Test1 you will get what you are looking for... — Grasshopper, Oct 23 '17 at 11:42
@Grasshopper thanks for the comment, even if i set the variable to non static it still skips the tests which are housed in the other class, any ideas? — xGIx, Oct 23 '17 at 11:45
What are you trying to achieve? Am I right that you're trying to skip all the remaining tests in a class if there is at least one test failed? — Alexey R., Oct 23 '17 at 11:56
@Alexey R.correct, Test1 has one failure therefore the remaining methods in that class should be skipped however Test2 has no failures therefore no methods should be skipped. — xGIx, Oct 23 '17 at 12:00
Check the import for `@BeforeClass`. It should use the annotation from testNG not from junit. (Please share the full code and any exceptions if it occurs for further analysis) — Syam Kumar S, Oct 23 '17 at 12:03
Have you tried to debug your beforeMethod() code? This might give you a clue why your numb==1. Try to set up some breakpoints and do some investigation. — Alexey R., Oct 23 '17 at 12:06
@ Alexey R thanks i have added the console output but cant see any issues? — xGIx, Oct 23 '17 at 12:08
Your @BeforeMethod is getting skipped for some reason. try this `@BeforeMethod(alwaysRun = true)` — Syam Kumar S, Oct 23 '17 at 12:12
@Syam Kumar thanks i have tried the alwaysRun=true but im afraid it still doesn't work — xGIx, Oct 23 '17 at 12:16
Please share your configuration xml file also. This can be caused by wrong configuration. — Syam Kumar S, Oct 23 '17 at 12:20

score 3 · Accepted Answer · answered Oct 23 '17 at 14:16

3

default behaviour for configFailurePolicy is "skip". Because if you get exception in configuration method then there is not need to run your test. But if you want to run your test anyway, change this configuration to "continue" It could be achieved by updating configuration

  <suite name="SuiteName" parallel="tests" thread-count="1" verbose="10" configfailurepolicy="continue">

answered Oct 23 '17 at 14:16

Zak

101
3

I had suites described across some xml-files and a parent suite that just lists suite-files. My test were skipped in the same manner because apparently `configfailurepolicy="continue"` is ignored if it is written in parent `testng.xml`, so it needs to be in the lowest level suite description. – sukhmel Nov 08 '18 at 20:25

Why is throw new SkipException() skipping all my methods?

1 Answers1

Linked