What happens in the background when testing a project in netbeans via Project->Menu->Test?

Question

I have a java project created on netbeans, the project has some test in project's test packages.

I have different behaviors when I test the project via Project->Menu->Test and test via creating junit4Suite with all class Test files.

For example, one difference is the velocity:

testing via suite: 58 test (all) takes 2.7 seconds
testing via Project->Menu->Test: 58 test(all) takes 14.9 seconds

What happens in background when netbeans is testing the project via Project->Menu->Test?

EDIT:

The project is very small so the time netbeans takes before execute testing are around 1 second.

Another diference:

I have created another dummy project with a method that you can only run once:

public class ScriptThreadWorker {

    private static ScriptThreadWorker worker;
    private Thread thread;

    public static ScriptThreadWorker getWorker() {
        if (worker == null) {
            worker = new ScriptThreadWorker();
        }
        return worker;
    }

    public void runOnScriptThread(Runnable task) {
        if (thread != null) {
            throw new RuntimeException("you can run once");
        }

        thread = new Thread(task);
        thread.setName("ui-thread");
        thread.start();
    }
}

And I have created 2 identical 2 test classes with a junit suite:

ScriptThreadWorker1Test.java

public class ScriptThreadWorker1Test {

    @Test
    public void testRun1() {
        ScriptThreadWorker.getWorker().runOnScriptThread(new Runnable() {
            @Override
            public void run() {
                // do nothing
            }
        });
    }
}

ScriptThreadWorker2Test.java

public class ScriptThreadWorker2Test {

    @Test
    public void testRun1() {
        ScriptThreadWorker.getWorker().runOnScriptThread(new Runnable() {
            @Override
            public void run() {
                // do nothing
            }
        });
    }
}

NewTestSuite.java

@RunWith(Suite.class)
@Suite.SuiteClasses({
    javaapplication3.ScriptThreadWorker2Test.class,
    javaapplication3.ScriptThreadWorker1Test.class})
public class NewTestSuite {
}

When I execute the test via Project->Menu->Test: "Both test passed"

But when I execute the test via NewTestSuite.java->Menu->Test file: "1 test passed, 1 test caused an error, you can run once".

score 0 · Answer 1 · answered Oct 21 '13 at 18:58

0

Netbeans like other IDEs makes the project before run. The time spent for synchronization resources and check the state of the files, compilation if necessary is spent for it by the IDE. If necessary it could be configured to perfom other tasks like ant run, maven build and so on.

answered Oct 21 '13 at 18:58

Thank you for the answer, the project is very small, so the time netbeans takes before execute the test is aprox. 1 second. – iberck Oct 21 '13 at 19:02
Netbeans also captured stdout and writes to it if your tests using logging for output to console then it will take a time. – Oct 21 '13 at 19:05
I have disabled logging, the diference is still about 10 seconds. – iberck Oct 21 '13 at 19:27

score 0 · Accepted Answer · edited May 23 '17 at 12:10

Answering my own question:

Netbeans runs the commands via ant, so in ${project_name}private/build-impl.xml are all the logic about project IDE commands.

For execute the unit test, the IDE uses the junit ant's task:

<macrodef name="junit" uri="http://www.netbeans.org/ns/j2se-project/3">
            <attribute default="${includes}" name="includes"/>
            <attribute default="${excludes}" name="excludes"/>
            <attribute default="**" name="testincludes"/>
            <attribute default="" name="testmethods"/>
            <element name="customize" optional="true"/>
            <sequential>
                <property name="junit.forkmode" value="perTest"/>
                <junit dir="${work.dir}" errorproperty="tests.failed" failureproperty="tests.failed" fork="true" forkmode="${junit.forkmode}" showoutput="true" tempdir="${build.dir}">
                    <test methods="@{testmethods}" name="@{testincludes}" todir="${build.test.results.dir}"/>
                    <syspropertyset>
                        <propertyref prefix="test-sys-prop."/>
                        <mapper from="test-sys-prop.*" to="*" type="glob"/>
                    </syspropertyset>
                    <formatter type="brief" usefile="false"/>
                    <formatter type="xml"/>
                    <jvmarg value="-ea"/>
                    <customize/>
                </junit>
            </sequential>
        </macrodef>

The interesting line is:

<property name="junit.forkmode" value="perTest"/>

From Junit Task documentation junit.forkmode:

Controls how many Java Virtual Machines get created if you want to fork some tests. Possible values are "perTest" (the default), "perBatch" and "once". "once" creates only a single Java VM for all tests while "perTest" creates a new VM for each TestCase class. "perBatch" creates a VM for each nested and one collecting all nested s. Note that only tests with the same settings of filtertrace, haltonerror, haltonfailure, errorproperty and failureproperty can share a VM, so even if you set forkmode to "once", Ant may have to create more than a single Java VM. This attribute is ignored for tests that don't get forked into a new Java VM. since Ant 1.6.2

Conclusion: Netbeans runs the junit test in forkMode=perTest, so it creates a new VM for each TestCase class, otherwise NewTestSuite only uses one VM.

For more detail about forkMode=perTest: junit: impact of forkMode="once" on test correctness

What happens in the background when testing a project in netbeans via Project->Menu->Test?

2 Answers2