1

I'm trying to run all tests in the project using testng.xml I have 80 classes with ~ 140 tests but every time I run test, testng runs only 83/140 tests. I tried to find pattern in tests that are not being executed but could not find any.

All tests/class files are under 'scripts' package and all class files have:

package scripts;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

All tests have @Test annotation (i.e. there are 140 @Test tags) @Test(priority=1,testName="",description="")

I am NOT using enabled="false" in @Test annotation

My testng.xml 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Regression">
    <test name="Regression">
        <packages>
            <package name="scripts"/>
        </packages>
    </test>
</suite>

Note: here around 57 tests are skipped for some reason. e.g. one of them is addMsd

  1. For some classes, if class has 3 tests, only 2 are executed
  2. For some classes, whole class file (having single or multiple tests) is skipped

If I add an additional parameter and add a test (that is usually skipped with above XML) as follows then it is executed successfully:

`<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Regression">
    <test name="Regression">
        <packages>
            <package name="scripts"/>
        </packages>
    </test>
    <test name="test">
        <classes>
            <class name="scripts.CalculateMSD">
                <methods><include name="addMsd"/></methods>
            </class>
        </classes>
    </test>
</suite>`

Here, addMsd will be executed successfully (which would be skipped in first xml)

I have tried running tests using IntelliJ testng plugin (right click > run) and maven surefire plugin and observed same behavior in both approaches.

I can't figure out what could be wrong with XML or the project setting. Any pointers will help!

Yogesh
  • 125
  • 2
  • 13
  • did you check the console ? does it contain any error ? – Sameer Arora Feb 19 '19 at 08:32
  • There is no error on console. But what I noticed is that most of the tests that run successfully are `priority=1` (or `priority=2`) and **ALL** the tests that are skipped are `priority = 2`. If I remove priorities for all tests, then all tests are executed !! I suspect it has something to do with the priorities (e.g. if priority 1 test fails in a class then don't execute priority 2 test?).... Does testng has any logic for skipping tests based on priority? – Yogesh Feb 20 '19 at 00:46

0 Answers0