1

I've come across a very strange problem when executing my QUnit tests in IE9. The test structure is something like this:

function setupTest() {
    // Adding some extra items to the DOM
}

function teardownTest() {
    // Removing dirtied items from the DOM
}

module("MODULE A", {setup: setupTest, teardown: teardownTest});

test("TEST 1", function () {
    // Some simple assertions
});

module("MODULE B", {setup: setupTest, teardown: teardownTest});

test("TEST 2", function () {
    // Some simple assertions
});

About 50% of the time these tests fail. I have made sure the tests are entirely independent and the setup/teardown fully completes before/after a test runs. The problem comes down to the setup callbacks being called out of sequence half of the time. With logging I can see this happening:

Successful run:

LOG: completing setup
LOG: MODULE A: TEST 1
LOG: completing teardown
LOG: completing setup
LOG: MODULE B: TEST 2
LOG: completing teardown

Unsuccessful Run:

LOG: completing setup
LOG: completing setup    <-- ?
LOG: MODULE B: TEST 2
LOG: completing teardown
LOG: MODULE A: TEST 1
LOG: completing teardown

As you can see on the unsuccessful run the setup callback is correctly executed twice, but before either of the tests are executed. Since teardown removes the required DOM elements the second executed test is bound to fail as its setup was called too early and the first executing test (correctly) ripped it out with its teardown.

This problem always occurs when the second test is executed before the first, but I can see no reason that this should cause a problem. The issue never appears in Chrome.

creimers
  • 4,975
  • 4
  • 33
  • 55
Jason
  • 2,271
  • 2
  • 24
  • 23
  • The tests aren't entirely independent. If they were you wouldn't have this problem. – Nathan MacInnes Jun 01 '12 at 08:52
  • Even if they weren't independent (which I assure you they are since I've cut them back to doing nothing other than testing the DOM structure is as they expect it after setup) then that doesn't explain why both tests have their setup callback called before any of the tests run, rather than before each test runs (as should happen). – Jason Jun 01 '12 at 09:23
  • It might or might not be related with this Question: http://stackoverflow.com/questions/11036514/qunit-async-tests-with-setup-and-teardown and this Pull Request for a test which might be a QUnit Bug: https://github.com/jquery/qunit/pull/320 – pscheit Sep 21 '12 at 14:36

1 Answers1

0

I had the same issue in Chrome (exactly 50% of the time my tests fail) and solved it in next way:

Code with bug:

require(["jquery", "jquerymobile", "QUnit", "underscore", 
    "units/general.unit", "units/mapping.unit"],
function( $, jqm, __QUnit, _ ) {
    var test_modules = Array.prototype.splice.call(arguments, 4);
    _.each( test_modules, function(test){
        test.start();
    });
    QUnit.load();
    QUnit.start();
});

Corrected version

require(["jquery", "jquerymobile", "QUnit", "underscore", 
    "units/general.unit", "units/mapping.unit"],
function( $, jqm, __QUnit, _ ) {
    QUnit.load();
    QUnit.start();
    var test_modules = Array.prototype.splice.call(arguments, 4);
    _.each( test_modules, function(test){
        test.start();
    });
});

QUnit.load() and QUnit.start() moved before the tests invocation and it do the trick.

polarfish
  • 85
  • 8