Run grouped tests sequentially using Playwright

Question

I'm using Playwright with nodejs and I have grouped a couple of tests together like this

import { test, expect } from '@playwright/test';

test.describe('Add a simple invoice test', () => {
    test('01.Login & add an invoice', async ({ page }) => {
        await page.goto("https://someUrl.com");
        await page.fill('input[id="email"]', "someEmailAddress");
        await page.fill('input[ng-model="ctrl.user.password"]', "somePassword");
        await page.click('button[id="login-btn"]');
    });

    test('02.Add an invoice', async ({ page }) => {
        await page.click('[name="invoice"]');
        await page.click('button[id="addInvoiceButton"]');
        await page.click('a[ng-click="ctrl.goToAddInvoice()"]');
        await page.fill('#invoiceTitle', Math.random().toString(36).substring(7));
        await page.fill('#exampleInputAmount', "120");
        await page.click("#invoiceCategory")
        await page.fill("#invoiceCategory > input", "Car")
        await page.keyboard.press("Enter");
        await page.click('button[id="submitInvoiceButton"]');
    });
});

The problem is that these 2 tests run in parallel whereas 02 is dependant on 01 since there's a required login.

How do I make 2 grouped tests run in the same context?

score 11 · Answer 1 · edited Feb 27 '23 at 23:52

The solution is very simple. It is executing as an independent test since you are passing {page} in each test, so if you want to use the same context for both the test you need to modify the test like below.

import { test, expect } from '@playwright/test';

test.describe('Add a simple invoice test', () => {
  let page: Page;
  test.beforeAll(async ({ browser }) => {
    page = await browser.newPage();
  });

    test('01.Login & add an invoice', async () => { // do not pass page
        await page.goto("https://someUrl.com");
        await page.fill('input[id="email"]', "someEmailAddress");
        await page.fill('input[ng-model="ctrl.user.password"]', "somePassword");
        await page.click('button[id="login-btn"]');
    });

    test('02.Add an invoice', async () => { //do not pass page 
        await page.click('[name="invoice"]');
        await page.click('button[id="addInvoiceButton"]');
        await page.click('a[ng-click="ctrl.goToAddInvoice()"]');
        await page.fill('#invoiceTitle', Math.random().toString(36).substring(7));
        await page.fill('#exampleInputAmount', "120");
        await page.click("#invoiceCategory")
        await page.fill("#invoiceCategory > input", "Car")
        await page.keyboard.press("Enter");
        await page.click('button[id="submitInvoiceButton"]');
    });
});

This should work as you expected You can also refer to the Dzone Article regarding the same.

Note: Playwright doesn't recommend this approach but this answer should fulfill your need.

Why not `await page.goto("https://someUrl.com");` inside beforeAll hook? — priyadarshi kumar, Jan 02 '22 at 16:03
I tried this. I'm now no longer getting traces generated for my test failures, which is a pretty big downside. Playwright has a test.step feature that I'm trying out as a replacement. — joewang704, Apr 01 '22 at 21:30
[Release 1.14](https://playwright.dev/docs/release-notes#version-114) uses `test.describe.serial('group', () => {` explained in chapter [serial mode with describe serial](https://playwright.dev/docs/release-notes#-serial-mode-with-describeserial) — surfmuggle, Mar 13 '23 at 14:09

score 6 · Answer 2 · answered Jun 16 '21 at 12:20

6

You shouldn't make your tests depend on each other. For example, I would extract the common login to a function and call that function from both tests. You could even add some asserts to the first test to check that the login worked. But you wouldn't do that on the second one.

answered Jun 16 '21 at 12:20

hardkoded

18,915
3
52
64

1

I know, what you're saying makes sense and I completely understand. The thing is I was just playing around with it to get to know the tool better. But knowing that in other testing frameworks like Jasmine for example, grouping some tests using the "it" keyword ( just like "test" here) helps divide the tests in understandable steps by everyone which are then logged to console. So I have found it a bit strange to behave like this. – Amrou Jun 16 '21 at 12:31
I don't want to sound rough, take this in a good sense :), but consider `it` or `test` as steps is wrong. every `it` should be independent, and you group it in files or `describes` because those tests are testing the same functionality. If you want to reuse steps or declare steps, those are just simple functions not `it`s. – hardkoded Jun 16 '21 at 13:19
I can't take something as rough when it's correcting something wrong that I'm doing. And I appreciate it a lot. I will apply what you told me in the future project. So just to see if I understood fully, if we suppose we want to test the login feature. The first **it** for example would be logging in with correct credentials and the second one with wrong credentials ? – Amrou Jun 16 '21 at 13:31
@amrou unfortunately, browsers/OSes have global state such as clipboard contents, so you'll get race conditions if testing such features in parallel. Agreed that in general you as a developer shouldn't INTRODUCE dependencies between tests. – d2vid Feb 22 '23 at 15:04

score 5 · Answer 3 · answered Feb 22 '23 at 15:20

Unfortunately, browsers/OSes have global state such as clipboard contents, so you'll get race conditions if testing such features in parallel.

If all your tests are running in parallel, you've probably enabled parallelism in playwright config:

const config: PlaywrightTestConfig = {
  fullyParallel: true,
  ...

This is good - you should leave fullyParallel: true to execute your tests faster, and then opt-out of running the tests in a single file (or single describe block) serially by adding test.describe.configure({ mode: 'serial' }); like this:

import { test, expect } from '@playwright/test';

test.describe('Add a simple invoice test', () => {
  test.describe.configure({ mode: 'serial' });

  test('01.Login & add an invoice', async ({ page }) => {
    await page.goto("https://someUrl.com");
    await page.fill('input[id="email"]', "someEmailAddress");
    await page.fill('input[ng-model="ctrl.user.password"]', "somePassword");
    await page.click('button[id="login-btn"]');
  });

  test('02.Add an invoice', async ({ page }) => {
    await page.click('[name="invoice"]');
    await page.click('button[id="addInvoiceButton"]');
    await page.click('a[ng-click="ctrl.goToAddInvoice()"]');
    await page.fill('#invoiceTitle', Math.random().toString(36).substring(7));
    await page.fill('#exampleInputAmount', "120");
    await page.click("#invoiceCategory")
    await page.fill("#invoiceCategory > input", "Car")
    await page.keyboard.press("Enter");
    await page.click('button[id="submitInvoiceButton"]');
  });
});

Read more in the playwright docs here: https://playwright.dev/docs/next/test-parallel#parallelize-tests-in-a-single-file

This helped me fix my problem. Thanks @d2vid – Libin Thomas Jul 22 '23 at 13:33 — Libin Thomas, Jul 22 '23 at 13:33

score 0 · Answer 4 · answered Jun 23 '21 at 22:40

0

If you have a bunch of tests that have common setup steps then Playwright Test Runner has some pre-canned features you can use to either run once before all the tests (test.beforeAll) or alternatively before each test (test.beforeEach).

Docs for that are here: https://playwright.dev/docs/test-intro#use-test-hooks

There are similar options for teardown, so they could be something simple like going to the login page and logging in (then logging out at the end?), or something more complex like setting up data and clearing it down after the test.

answered Jun 23 '21 at 22:40

sahmeepee

378
2
8

1

What actually want is to execute the tests in a BDD manner, with a description of each step being execution shown to the console. Because some people sometimes track the tests & don't know what's going on. So I want to show something like this: 1. User logs in. 2. Navigates to menu X. 3. Does Y. 4. Logs out – Amrou Jun 24 '21 at 09:22
I'm not 100% clear whether you want to record that an action is being taken (you have controlled the browser) or that the action has worked as intended (the result you expect is found in the DOM) If it's the former, you can just dump output into the console with: console.log(blah); For the latter you can use assertions ("expect"). – sahmeepee Jun 26 '21 at 14:32

score 0 · Answer 5 · answered Nov 07 '22 at 21:25

You can split the file and run it separately too.

test1.ts

import { test, expect } from '@playwright/test';

test('01.Login & add an invoice', async ({ page }) => {
    await page.goto("https://someUrl.com");
    await page.fill('input[id="email"]', "someEmailAddress");
    await page.fill('input[ng-model="ctrl.user.password"]', "somePassword");
    await page.click('button[id="login-btn"]');
});

test2.ts

import { test, expect } from '@playwright/test';

test('02.Add an invoice', async ({ page }) => {
    await page.click('[name="invoice"]');
    await page.click('button[id="addInvoiceButton"]');
    await page.click('a[ng-click="ctrl.goToAddInvoice()"]');
    await page.fill('#invoiceTitle', Math.random().toString(36).substring(7));
    await page.fill('#exampleInputAmount', "120");
    await page.click("#invoiceCategory")
    await page.fill("#invoiceCategory > input", "Car")
    await page.keyboard.press("Enter");
    await page.click('button[id="submitInvoiceButton"]');
});

And then run it like:

npx playwright test test1.ts

npx playwright test test2.ts

or depending on your setting it could also be like:

pnpm -C test test1.ts

pnpm -C test test2.ts

score 0 · Answer 6 · answered Nov 29 '22 at 07:14

0

the dzone article referenced in the checked answer uses typescript, but if you don't use typescript here's an example you can use on the playwright docs: https://playwright.dev/docs/test-parallel#serial-mode

answered Nov 29 '22 at 07:14

michelle_c

1

1

Please add the relevant code directly to your answer. – Ingo Steinke Dec 05 '22 at 10:09

Run grouped tests sequentially using Playwright

6 Answers6