TestNG - Weird behavior

Question

I am little bit confused by this testNG behavior.

Consider this simple testNG suite. Test2 depends on Test1. The below suite also starts Test2 only after Test1 which is great!

<suite name="testng-behvaior" parallel="none">

    <test name="test1">
        <parameter name="browser" value="chrome" />
        <classes>
            <class name="com.test.Test1" />
            <class name="com.test.Test2" />
        </classes>
    </test>

</suite>

Above suite works just fine without any issues.

But consider this. As per testNG documentation, all the <test> would be assigned to different thread. Since i have only one <test>, only one thread is executing this suite which is fine. But it starts with Test2 before Test1.

   <suite name="testng-behvaior" parallel="tests">

        <test name="test1">
            <parameter name="browser" value="chrome" />
            <classes>
                <class name="com.test.Test1" />
                <class name="com.test.Test2" />
            </classes>
        </test>

    </suite>

For me - in the above cases, parallel="tests" and parallel="none" should not make any difference and behave same.

What makes testNG should behave differently? How can i have the thread to execute the <classes> within the <test> in sequence?

score 0 · Accepted Answer · answered Jan 10 '19 at 04:36

0

There is an attribute called preserve-order. By default this attribute has a value of true at both the <test> level and at the <suite> level.

This attribute when enabled, causes TestNG to load classes in the same order as defined in the <test> tag and then run all the @Test methods defined within each of them.

But for this attribute to work, you would need to disable parallelism i.e., you need to set the attribute parallel to none.

Only then does TestNG execute the tests in the order in which they are found in the <test> tag.

So parallel="tests" is NOT EQUAL TO parallel="none"

answered Jan 10 '19 at 04:36

Krishnan Mahadevan

14,121
6
34
66

So you basically mean this is how testng behaves which could be as designed for you guys but this is wrong! – KitKarson Jan 10 '19 at 14:43
TestNG does not care if you had only 1 `` tag or if you had many `` tags. The moment you choose your parallelism strategy via `parallel` attribute the `preserve-order` functionality is disabled. And lets say you aren't happy with the behavior (which is what it looks like from your response), you should be logging a bug in the TestNG github issues page and debate/discuss it there. My answer was just to clarify to you why are you seeing the behavior, not to justify why it is so. Hope that adds clarity. – Krishnan Mahadevan Jan 10 '19 at 15:14
I see.. Thanks for the explanation. – KitKarson Jan 10 '19 at 16:08
Yes. This ensures that the `` doesn't inherit the parallel attributes from the `` tag. – Krishnan Mahadevan Jan 10 '19 at 16:32

score 0 · Answer 2 · answered Jan 10 '19 at 16:15

For those who has been facing similar issue - if you want the tests to be parallel and but want to execute the classes within the test in the sequence, use the below approach

   <suite name="testng-behvaior" parallel="tests">

        <test name="test1" parallel="none"> <!-- we do not want parallel here -->
            <parameter name="browser" value="chrome" />
            <classes>
                <class name="com.test.Test1" />
                <class name="com.test.Test2" />
            </classes>
        </test>

    </suite>

TestNG - Weird behavior

2 Answers2