0

Can anyone tell me how to get (expected) count of all tests / method executions at TestNG (suite) start?

You can easily get the amount of all test methods, but this doesn't count each test method times the amount of the data provider size (each method will be counted once, even if one method is executed 100 times via a data provider).

Unless there is an obvious solution to this, like a simple API call, the key seems to be to figure out the data provider method from the data provider name given in the @Test annotation, which I asked here: How to get data provider method from TestNG data provider name?

Once you know what the data provider method is, you could execute the data provider, and count the amount of data sets. This doesn't seem very efficient, but the tests we're running are not unit tests, so I wouldn't be too worried about executing the data providers twice (it will be a peanut compared to how long the actual tests run).

Community
  • 1
  • 1
mac
  • 2,672
  • 4
  • 31
  • 43

3 Answers3

0

You can use a listener and having your own count logic inside.

juherr
  • 5,640
  • 1
  • 21
  • 63
  • Sure, but how would that count logic look? – mac Jul 18 '15 at 07:28
  • Well, in fact I'm not sure to understand your problem. For example, what is your expected count for https://github.com/cbeust/testng/blob/master/src/test/java/test/dataprovider/ClassSampleTest.java ? – juherr Jul 21 '15 at 10:50
  • 4 cause 2 test methods x 2 entries from the data provider – mac Jul 21 '15 at 17:28
  • I didn't catch it before but you want the count information **BEFORE** the tests execution? What is the purpose? – juherr Jul 22 '15 at 09:21
  • Yup, before the test execution. We have some logic with RetryAnalyzers. I don't want the total amount of retries to be higher than a given percentage (e.g. 10% or 20%). To calculate that, I need to know what the total amount of tests is before the first test runs. – mac Jul 23 '15 at 14:31
  • Currently, TestNG itself only know data provider content when it executes tests. Could you create an issue on Github about this need (and another one for your other question/need). We can continue to discuss about it there. Then, I will delete my answer as it finally doesn't answer the question. – juherr Jul 23 '15 at 19:21
0

Unfortunately, there is no easy way of getting either of below directly in TestNG-

  1. no. of test cases passed
  2. Success Percentage

Here's the workaround, I did to get PASS percentage. As you can see that run summary prints the below including count of Dataprovider tests-

Total tests run: 4, Passes: 1, Failures: 3, Skips: 0

so Total test case = Pass count + Fail count + Skip Count So I wrote a method like this-

public static void main(String[] args) {
    // TODO Auto-generated method stub
    System.out.println("###################  TESTS STARTED   ##################### ");
            TestListenerAdapter tla = new TestListenerAdapter();
            TestNG testng = new TestNG();
            testng.setTestClasses(new Class[] { PassCSVDatatoDataProvider.class });
            testng.addListener(tla);
            testng.run();
            
            float Totalcount = (tla.getFailedTests().size()+tla.getPassedTests().size()+tla.getSkippedTests().size());
            float Passpercentage = (tla.getPassedTests().size()/Totalcount*100);
            System.out.println("PASS PERCENTAGE - "+Passpercentage +"%");
            System.out.println(tla.getFailedTests().size()+ "  TESTS FAILED ! ");
            
            System.out.println("###################      END   ##################### ");
        }

TestNG Console Output

###################  TESTS STARTED   ##################### 
=============================================== Command line suite Total tests run: 4, Passes: 1, Failures: 3, Skips: 0
===============================================

PASS PERCENTAGE - 25.0% 
3  TESTS FAILED ! 
###################      END   #####################
Fukasi
  • 1
  • 2
0

In dry run try to implement IInvokedMethodListener and override beforeInvocation method

public class Test implements IInvokedMethodListener
{
    static int testcount=0;
    public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
            
            testcount=testcount+method.getTestMethod().getInvocationCount();
    }

    @Override
    public void onStart(ISuite suite) {
        // TODO Auto-generated method stub
        
    }

    
    @Override
    public void onFinish(ISuite suite) {
        System.out.println(testcount);
        
    }
}
John Conde
  • 217,595
  • 99
  • 455
  • 496
vijayv
  • 1