Retrying a complete testNG class and not just @Test method on skip/failure

Question

I have implemented IRetryAnalyzer to re-run my failed test cases in my testNG class.

public class Retry implements IRetryAnalyzer {
    private int retryCount = 0;
    private int maxRetryCount = 1;
    private int outcome;

// Below method returns 'true' if the test method has to be retried else 'false' 
//and it takes the 'Result' as parameter of the test method that just ran
    public boolean retry(ITestResult result) {
     //outcome=result.getStatus();
        if (retryCount < maxRetryCount ) {
         result.getTestContext().getFailedTests().removeResult(result);
         result.getTestContext().getSkippedTests().removeResult(result);
            System.out.println("Retrying test " + result.getName() + " with status "
                    + getResultStatusName(result.getStatus()) + " for the " + (retryCount+1) + " time(s).");
            Reporter.log("Retrying test " + result.getName() + " with status "
                    + getResultStatusName(result.getStatus()) + " for the " + (retryCount+1) + " time(s).");
            
            retryCount++;
            return true;
        }
        return false;
    }
    
    public String getResultStatusName(int status) {
     String resultName = null;
     if(status==1)
      resultName = "SUCCESS";
     if(status==2)
      resultName = "FAILURE";
     if(status==3)
      resultName = "SKIP";
  return resultName;
    }
}

Now I have two Test methods:

@Test(priority = 3, enabled = true, dependsOnMethods={"step2"})
 public void step3() 
  
{.....................some code......}

@Test(priority = 4, enabled = true,dependsOnMethods={"step3"})
 public void step4() {
  
  ....some codee..}

If step 3 fails, testNG skips step 4 which is as expected. But upon re-run it executes only step 3 and even if it passed at second attempt, step 4 which was skipped is not executed.

Is there any way I can re-run my whole TestNG failed class or an alternate solution to run my dependent cases after the @Test method they depend on fails.

Thanks in advance!

Answer 1

Please do the following to get this to work:

1. Remove the logic of removing failed and skipped tests from your org.testng.IRetryAnalyzer implementation i.e., the below two lines

result.getTestContext().getFailedTests().removeResult(result); result.getTestContext().getSkippedTests().removeResult(result);

2. Include this logic of removing the skipped/failed tests from within either an @AfterMethod method (or) from within an afterInvocation() of a org.testng.IInvokedMethodListener listener implementation.

Something like below :

@AfterMethod
public void afterMethod(ITestResult result) {
    IRetryAnalyzer retry = result.getMethod().getRetryAnalyzer();
    if (retry == null) {
        return;
    }
    result.getTestContext().getFailedTests().removeResult(result.getMethod());
    result.getTestContext().getSkippedTests().removeResult(result.getMethod());
}

(or)

import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class Listener implements IInvokedMethodListener {
    @Override
    public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {

    }

    @Override
    public void afterInvocation(IInvokedMethod method, ITestResult result) {
        IRetryAnalyzer retry = result.getMethod().getRetryAnalyzer();
        if (retry == null) {
            return;
        }
        result.getTestContext().getFailedTests().removeResult(result.getMethod());
        result.getTestContext().getSkippedTests().removeResult(result.getMethod());
    }
}

If you leverage the listener path, please ensure that you wire in the listener using one of the following ways :

1. via @Listeners annotation (or)
2. via <listeners> tag (or)
3. via service loaders in TestNG.

For more information refer to my blog post here.

Answer 2

Additing to the Krishnan Mahadevan answer, you can chose to skip

result.getTestContext().getFailedTests().removeResult(result.getMethod());

If you remove a failed test method then, depending on test case (step4) will be executed even when step3 gets fail after retry.