Run test based on defined priority with multiple classes through test.xml file

Question

I have 3 classes for with 3 tests each.

Class 1

@Test( priority = 1 )
public void testA1() {
    System.out.println("testA1");
}

@Test( priority = 2 )
public void testA2() {
    System.out.println("testA2");
}

@Test( priority = 3 )
public void testA3() {
    System.out.println("testA3");
}

Class 2

@Test( priority = 1 )
public void testB1() {
    System.out.println("testB1");
}

@Test( priority = 2 )
public void testB2() {
    System.out.println("testB2");
}

@Test( priority = 3 )
public void testB3() {
    System.out.println("testB3");
}

Class 3

@Test( priority = 1 )
public void testC1() {
    System.out.println("testC1");
}

@Test( priority = 2 )
public void testC2() {
    System.out.println("testC2");
}

@Test( priority = 3 )
public void testC3() {
    System.out.println("testC3");
}

This is my XML file code.

<test verbose="2" name="hello" group-by-instances="true">
    <classes>
        <class name="Class1"></class>
        <class name="Class2"></class>
        <class name="Class3"></class>
    </classes>
</test>

Here is my Answer testA1 testB1 testC1 testA2 testB2 testC2 testA3 testB3 testC3

But my expected answer is testA1 testA2 testA3 testB1 testB2 testB3 testC1 testC2 testC3

Thanks in advance for any help on this.

I am running through maven framework and directly in intellij, same error for both cases — Bhavik Shah, Feb 24 '17 at 12:38

juherr · Answer 1 · 2017-02-24T13:28:03.927

1

The seen behavior is the expected one.

In fact, priority is more important that group-by-instances ( https://github.com/cbeust/testng/blob/master/CHANGES.txt#L48) and that's why TestNG respect priority instead of group-by-instances.

To achieve your expected behavior, you have to replace priority by a more important order feature, like dependsOnMethods:

@Test
public void testA1() {
    System.out.println("testA1");
}

@Test( dependsOnMethods = "testA1" )
public void testA2() {
    System.out.println("testA2");
}

@Test( dependsOnMethods = "testA2" )
public void testA3() {
    System.out.println("testA3");
}

As asked in the comments, if you really want a "priority on a class without a strong dependency", you can make it yourself with a method interceptor where you can order methods as you want. In pseudo code, something like:

public class PriorityOnClassOrder implements IMethodInterceptor {

  public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {
    // 1. Group by instance/class
    Map<Class<?>, List<IMethodInstance>> map = ...
    for (IMethodInstance method : methods) {
      map.get(method.getInstance().getClass()).add(method);
    }

    List<IMethodInstance> result = ...
    // 2. Order methods from an instance/clas according to their priority
    for(Map.Entry entry : map.entries()) {
      List<IMethodInstance> m = entry.value();
      Collections.sort(m, new Comparator<IMethodInstance>() {
        public int compare(IMethodInstance o1, IMethodInstance o2) {
          return o1.getMethod().getPriority() - o2.getMethod().getPriority()
        }
      });
      result.addAll(m);
    }

    // 3. Return the result
    return result;
  }
}

edited Feb 24 '17 at 13:28

answered Feb 24 '17 at 12:38

juherr

5,640
1
21
63

Thanks for your inputs. But i want to run independent tests. Depends on method will not work for me in this case. Is there any other way to sort it out with this priority numbers and xml file? – Bhavik Shah Feb 24 '17 at 12:45
Nope. Remove priority in that case. – juherr Feb 24 '17 at 12:47
I have refer this link for same issue. "http://stackoverflow.com/questions/26632241/priority-in-testng-with-multiple-classes". – Bhavik Shah Feb 24 '17 at 12:49
can we have class priority higher which is higher that test priority? Please help me on this. I am stuck from so long – Bhavik Shah Feb 24 '17 at 12:55
The answer from the other question worked before the change. You can an old TestNG version if you don't care about fixes. I will update my answer with a solution which can help you. – juherr Feb 24 '17 at 13:02
Yes actually i dont have any major issues with fixes. So i can use old version, i will try to update same in my code. – Bhavik Shah Feb 24 '17 at 13:05
You can try the proposed solution if you can. – juherr Feb 24 '17 at 13:28
I appreciate your efforts, but it looks so complex to me. I am looking for some easy way to sort it out... – Bhavik Shah Feb 24 '17 at 17:11

score 0 · Answer 2 · answered Feb 28 '17 at 08:31

0

As you probably noticed, priority flag affect for whole not for single class. The easiest way is to jus increase priority level in second class.

@Test( priority = 4 )
public void testA1() {
    System.out.println("testA1");
}

@Test( priority = 5 )
public void testA2() {
    System.out.println("testA2");
}

@Test( priority = 6 )
public void testA3() {
    System.out.println("testA3");
}

Also you can put single class in single , i think it's even better if you want to separate tests domain.

 <test name="Test1" verbose="3" >
  <classes>
    <class name="tests.NewTest"></class>
  </classes>
  </test> <!-- Test -->
  <test name="Test2" verbose="3" >
   <classes>
    <class name="tests.NewTest2"></class>
  </classes>
  </test>

And verbose flag. I hardly recommend this during debugging.

answered Feb 28 '17 at 08:31

underwater ranged weapon

444
6
18

In first case it will be really difficult for me to manage to give priority to every single test.. As total i will be having 400-500 Test cases. So this will not use for me. – Bhavik Shah Feb 28 '17 at 09:55
Second case is much better to use. But I have one more option which i added here.. – Bhavik Shah Feb 28 '17 at 09:56
You said that you have 400-500 test cases, and in your case, the result which you publish below you will need to include all methods, one by one. Maybe you should think about resolve made programmatic. Just make TestNG object and add classes method etc. by XmlSuite, XmlClass, XmlTest, then aggregate it into main object. – underwater ranged weapon Feb 28 '17 at 10:32

score 0 · Answer 3 · answered Feb 28 '17 at 10:01

Finally I got solution of problem.

Please refer below XML code for same.

<classes>
            <class name="Class1">
                <methods>
                    <include name="testA1" />
                    <include name="testA2" />
                    <include name="testA3" />
                </methods>
            </class>
           <class name="Class2">
                <methods>
                    <include name="testB1" />
                    <include name="testB2" />
                    <include name="testB3" />
                </methods>
            </class>
            <class name="Class3">
                <methods>
                    <include name="testC1" />
                    <include name="testC2" />
                    <include name="testC3" />
                </methods>
            </class>

This will give you expected result. Plus it will be easier to manage also. As if we want to see my all tests or remove some tests, i can look into test.xml

Run test based on defined priority with multiple classes through test.xml file

3 Answers3

Linked