Is jasmine supposed to execute specs in the order they are declared or in a random order?

Question

un-comment the last spec. All hell breaks loose... why?

describe('test', function() {
  var index = 1;

  it('test 1', function() {
    expect(index).toBe(1);
    index++;
  });

  it('test 2', function() {
    expect(index).toBe(2);
    index++;
  });

  it('test 3', function() {
    expect(index).toBe(3);
    index++;
  });

  it('test 4', function() {
    expect(index).toBe(4);
    index++;
  });

  it('test 5', function() {
    expect(index).toBe(5);
    index++;
  });

  it('test 6', function() {
    expect(index).toBe(6);
    index++;
  });

  it('test 7', function() {
    expect(index).toBe(7);
    index++;
  });

  it('test 8', function() {
    expect(index).toBe(8);
    index++;
  });

  it('test 9', function() {
    expect(index).toBe(9);
    index++;
  });

  it('test 10', function() {
    expect(index).toBe(10);
    index++;
  });

  // it('test 11', function() {
  //   expect(index).toBe(11);
  //   index++;
  // });

});

thanks to @PWKad for pointing out this happens when there are more than 10 tests.

It's always ran in the declared order for me. My only suggestion is to cut down the real suite to the bare minimum that shows the issue and post that code in the question. — Michal Charemza, May 05 '15 at 11:33
I don't have a huge amount of experience with jasmine, but with unit tests in general, you shouldn't expect them to run in any particular order - they're _unit_ tests after all. If a particular test needs certain conditions, that should be in the set up/tear down of that test. — James Thorpe, May 05 '15 at 11:35
@JamesThorpe the issue is the functionality being tested is stateful and has an asynchronous API. If I can't depend on the order specs are executed I'll end up with one deeply nested spec, which is ugly and doesn't reflect the parts of the API that are being tested. I can live with it if someone knows for sure what jasmine's "expected" behavior is. Up until now I've had the same experience as Michal Charemza. — Jeremy Danyow, May 05 '15 at 11:45
I would expect that the framework authors would tell you not to rely on the order of the tests as this is an implementation detail which may change with any release. I appreciate your comment above, but I suspect that relying on execution order will give you pain at the expense of not having 'ugly' tests. Usually your specs can have a 'background' where you can do your setup. — Sam Holder, May 05 '15 at 12:07
@SamHolder Agreed but there is something about having more than 10 tests that is triggering the random order. Why 10+? — PW Kad, May 05 '15 at 12:54
Running it with all 11 specs included at http://tryjasmine.com/ works fine for me. Must be jasmine-version specific. I agree it would be nice to know what's causing this, but I still wouldn't rely on the specific order of tests. — James Thorpe, May 05 '15 at 13:14

score 19 · Accepted Answer · answered Jul 24 '15 at 22:37

19

Yes, Jasmine executes the specs (it) in order. There was an issue from 2.3.0 to 2.3.3 with more than 10 specs. I hit the same issue in 2.3.3, the issue is fixed in 2.3.4.

https://github.com/jasmine/jasmine/issues/850

I just used 2.3.4 in place of 2.3.3 and my 15 tests finally passed.

answered Jul 24 '15 at 22:37

user1559679

306
3
4

8

This is now not the default, as @Anton says, Jasmine 3.x now run's tests in a random order by default. – Tom Jul 20 '18 at 09:42
5

So is there a way to turn it off in 3.x? – juniper- Sep 05 '18 at 19:15
1

Is there any way to disable this default behavior ? – Jessy Sep 09 '19 at 12:21

Anton · Answer 2 · 2017-02-24T06:35:39.360

11

Currently (v2.x) Jasmine runs tests in the order they are defined. However, there is a new (Oct 2015) option to run specs in a random order, which is still off by default. According to the project owner, in Jasmine 3.x it will be converted to be the default.

References:

edited Feb 24 '17 at 06:35

answered Jan 03 '16 at 17:11

Anton

1,560
18
29

score 2 · Answer 3 · answered Jul 18 '21 at 04:45

Here I am now in 2021, and indeed, the default setup via npx jasmine init sets random test order by default it seems.

Surely not what most developers would expect. (Not what I expected!)

To run in declared order, go into your spec/support/jasmine.json and set:

"random": false

Is jasmine supposed to execute specs in the order they are declared or in a random order?

3 Answers3

Linked