36

When I press the "run all specs" button or use the run command that runs all files in Cypress it runs all test files alphabetically, so I don't want that.

I want to sort all of them with my own rules.

Let's say I have 3 steps in a chat app test.

  1. Can connect the chat app
  2. Can connect the chat
  3. Can the user send a message

I want to test every step without being tied to each other. What I mean, Testing one of their own function. What I do is as follows

chat_app_connect.spec.js

describe('Server Connecting Test', () => {
    it('Visit Server page', () => {
        cy.visit('https://chat.page..');
    });

    it('Check welcome messages', () => {
        cy.contains('Live Support');
        cy.contains('Hello, Stranger');
    });

    it('Check URL and status of circle', () => {
        // URL
        cy.url()
            .should('include', '/hello');
        // Status Circle    
        cy.get('circle')
            .should('have.class', 'positive');
    });
});

chat_connect.spec.js

import './chat_app_connect.spec.js';

describe('Chat Connecting Test', () => {
    it('Type customer name', () => {
        cy.get('input')
            .clear()
            .type('E2E Test');
    });
    it('Click to the submit button', () => {
        cy.get('.submit-button')
            .click();
    });
    it('Check URL and status of circle', () => {
        // URL
        cy.url()
            .should('equal', 'https://client.dev.octopus.chat/');
        // Status Circle
        cy.get('circle', { timeout: 5000 })
            .should('have.class', 'positive');
    });
});

chatting.spec.js

import './chat_connect.spec.js';

describe('Chatting Tests', () => {
    it('Type a test message then press Enter and check the message if it sent', () => {
        // Type
        cy.get('#chat-message')
            .clear()
            .type('Hey I\'m a test message{enter}');
        // Check the message
        cy.get('.message-list')
            .should('contain', 'Hey I\'m a test message');
    });
});

as you see every test is tied to each other, and that is mean when I tried to test just catting functionality its call every test and the whole tests will be tested.

I don't know if it is the right way or not.

what should I do in this case or can it be an acceptable way

WouX
  • 363
  • 1
  • 3
  • 5

6 Answers6

50

I have a particular case where I launch multiple instances of an app, rather than using fixtures or test data, I simply integrate user feedback as Cypress tests from login on forwards.

In any case, I used the specPattern configuration property in cypress.json to set the spec file run order:

{
  "baseUrl": "http://localhost:5000",
  "specPattern": [
    "login/*.js",
    "leads/new-lead.spec.js",
    "leads/leads-list.spec.js",
    "leads/lead-detail.spec.js",
    "leads/lead-modify.spec.js",
    //...
  ]
}

No file numbering needed :D

This configuration property was added in version 10, see also:

hc_dev
  • 8,389
  • 1
  • 26
  • 38
JeanLescure
  • 1,029
  • 9
  • 19
  • 2
    That's will be very useful. I can get rid of the numbers. – WouX Feb 10 '20 at 11:30
  • 8
    Heads up on this: This (any any method of test ordering) is a by-product of the way that functionality works. The purpose of `testFiles` is to include only certain specs, not for ordering. That just happens to be the way it works internally and that may or may not continue to work forever, as Cypress makes no claims for it to work that way. – Brendan Feb 27 '20 at 16:47
  • 1
    A downside to this is that every new test file has to be added to `cypress.json` manually. – alcfeoh Jul 10 '20 at 22:54
  • 7
    Not a downside if you use the `*` pattern, and store your tests in such a way all those that aren't explicit are still executed at last – Vadorequest Jul 12 '20 at 07:56
  • can you post the directory tree you use this with? for some reason it can't find the folder i have my specs in. Thanks. – ml242 Mar 30 '22 at 22:54
  • 1
    For the record, according to the doc, they have renamed it to `specPattern`: https://docs.cypress.io/guides/references/configuration#Testing-Type-Specific-Options – Guillaume Bailly Sep 23 '22 at 07:04
  • 1
    As of June 2023 (Cypress 12) this has to be in ```cypress.config.ts``` instead of ```cypress.json```. Keep in mind that the specPattern has to be inside of ```e2e``` or in ```component``` respectively, which kind of tests you want to order. Also for me it was necessary to add ```**/``` in front of my test names, e.g. ```specPattern: ['**/login.cy.ts', '**/logout.cy.ts']. – MikhailRatner Jun 07 '23 at 08:27
17

The easiest solution is most likely to add a prefix to all your test files, such as:

  • 01-chat_app_connect.spec.js
  • 02-chat_connect.spec.js

etc.

Cypress is going to take those files in alphabetical order, which you can "trick" into your wanted behavior by using a number as a prefix.

alcfeoh
  • 2,147
  • 1
  • 18
  • 33
6

Cypress does not intentionally let you do this, and for good reasons:

  1. It's generally indicative of poor test design. Tests should not depend on the state of one another. Any test should be able to be run successfully in isolation from the rest of the test suite.
  2. You'll never be able to take advantage of cypress' built in ability to run tests in parallel since you can't guarantee one spec will be ran after another

Here is a relevant discussion about this that gets into more detail: https://github.com/cypress-io/cypress/issues/390

However, if you decide to do this anyway, you can do it by prefixing the name of the specs with a number:

01-some-spec.js
02-alphabetically-first-spec.js
03-some-other-spec.js
Brendan
  • 4,327
  • 1
  • 23
  • 33
  • I'm writing tests for a chat app, so the cases like "firstly login then start chatting", it's a little bit challenging. I do not want big code blocks, I want small code blocks in small files. I figured out that with imports but that doesn't seem right. – WouX Nov 20 '19 at 07:34
  • 3
    What I did in my tests is write a custom command that logs in using `cy.request()`, then in the `before()` block, call my `cy.login()` command. that is how you should set up state to avoid having to rely on a specific test execution order. – Brendan Nov 20 '19 at 19:48
6

Jean Lescure's answer was a lifesaver. We needed to run tests based on priority without having a bunch of duplicated tests or symlinks. The following worked for us in our default cypress config file:

  "integrationFolder":"cypress/integration",
  "testFiles": [
    "high_priority_specs/**/*.js",
    "medium_priority_specs/**/*.js",
    "low_priority_specs/**/*.js"
  ]

To change the level of priority we used 3 configs files that were loaded using the cypress --configFile argument. To run the higher priority tests (smoke tests only) we used the following:

  "integrationFolder":"cypress/integration",
  "testFiles": [
    "high_priority_specs/**/*.js"
  ]
Peter Drinnan
  • 4,344
  • 1
  • 36
  • 29
  • 1
    Thank you! For reference, those folder designations are relative to/children of `tests/e2e/specs`, so in Peter Drinnan's example here, the tests would presumably be organized as: `tests/e2e/specs/medium_priority_specs/.js` `tests/e2e/specs/high_priority_specs/.js` etc – aleph_one Oct 24 '21 at 21:53
1

The accepted answer is not up to date yet. For Cypress 12 you can configure the order of your tests inside of the cypress.config.ts file (instead of cypress.json) like this:

import { defineConfig } from 'cypress';

export default defineConfig({
    e2e: {
        baseUrl: 'http://localhost:5000',
        specPattern: [
            '**/login.cy.ts',
            '**/logout.cy.ts',
        ],
    },

    component: {
        devServer: {
            framework: 'angular',
            bundler: 'webpack',
        },
        specPattern: [
            '**/myFirstComponent.cy.ts',
            '**/mySecondComponent.cy.ts'
        ],
    },
});

Keep in mind that the specPattern has to be inside of e2e or in component depending on what kind of tests you want to order.

Also for me it was necessary to add **/ in front of my test names, e.g. '**/login.cy.ts'.

MikhailRatner
  • 452
  • 6
  • 13
0

In addition to @Brendan answer, if you have a nested folder structure, this approach will work as well.

01-folder-name
 |
  - 01-some-spec.js
Vlad R
  • 1,851
  • 15
  • 13