0

I am using codeceptjs library in order to develop and run automated tests. Currently I'm in process of run those tests in paralell using NodeJS` workers. CodeceptJS proposes this opportunity and mention it in its documentation - https://codecept.io/parallel/#parallel-execution-by-workers.

Also I'm using such reporters as the Mochawesome, mocha-junit-reporter and codeceptjs-cli-reporter that I can use all-in-one with the help of Mocha-multi package.

Command which I use in order to run my tests is

codeceptjs run-workers --suites 4 --reporter mocha-multi

My codecept.conf.js file looks like that:

    // Инициализация расширения dotenv, чтобы переменные из файла .env были доступны в тестах через объект process.env
require('dotenv').config();

const { setHeadlessWhen } = require('@codeceptjs/configure');
const { devices } = require('playwright');
const { BASE_URL, API_URL } = require('./utils/endpoints');

// Для запуска тестов в headless-режиме (как в bamboo) выполняется команда
// yarn e2e:ci
// Для запуска тестов в окне браузера выполняется команда
// yarn e2e
setHeadlessWhen(process.env.HEADLESS === 'true');

exports.config = {
    tests:   './e2e/*/*.test.js',
    output:  './output',
    helpers: {
        Playwright: {
            url:               `https://${BASE_URL}`,
            show:              true,
            browser:           'chromium',
            waitForNavigation: 'domcontentloaded',
            waitForTimeout:    3000,
            getPageTimeout:    10000,
            emulate:           devices['iPhone 6'],
        },
        REST: {
            endpoint: `https://${API_URL}`,
        },
        Auth: {
            require: './helpers/auth.js',
        },
        DataGenerator: {
            require: './helpers/data-generator.js',
        },
        Cabinet: {
            require: './helpers/cabinet.js',
        },
        Moderation: {
            require: './helpers/moderation.js',
        },
        Advert: {
            require: './helpers/advert.js',
        },
        User: {
            require: './helpers/user.js',
        },
        Faker: {
            require: './helpers/faker.js',
        },
        ChaiWrapper: {
            require: 'codeceptjs-chai',
        },
        Mochawesome: {
            uniqueScreenshotNames: true,
        },
    },
    include: {
        I:              './steps_file.js',
        // pages
        SubmitPage:     './pages/kaspi/Submit.js',
        IndexPage:      './pages/kaspi/Index.js',
        AdvertPage:     './pages/kaspi/AdvertPage.js',
        EditAdvertPage: './pages/kaspi/EditAdvert.js',
        CabinetActive:  './pages/kaspi/CabinetActive.js',
        // steps
        EditAdvertStep: './steps/kaspi/EditAdvert.js',
        AdvertPageStep: './steps/kaspi/AdvertPage.js',
        CabinetStep: './steps/kaspi/Cabinet.js',
    },
    bootstrap: null,
    mocha:     {
        reporterOptions: {
            'codeceptjs-cli-reporter': {
                stdout:  '-',
                options: {
                    verbose: true,
                    steps:   true,
                },
            },
            mochawesome: {
                stdout:  '-',
                options: {
                    reportDir:      './output',
                    reportFilename: 'report',
                },
            },
            'mocha-junit-reporter': {
                stdout:  '-',
                options: {
                    mochaFile:   './output/report.[hash].xml',
                    attachments: true,
                },
            },
        },
    },
    name:    'market-spa',
    plugins: {
        pauseOnFail:     {},
        retryFailedStep: {
            enabled: true,
        },
        tryTo: {
            enabled: true,
        },
        screenshotOnFail: {
            enabled: true,
        },
    },
};

The problem is that when I get an HTML report created by Mochawesome it contains only results of the last NodeJS` worker. In other words, I expect that my HTML report will contain results of all of my 20 automated tests, but it only contains the results of 5 tests (those 15 tests were also running, but left no results).

By the way, the xml results that mocha-junit-reporter generates are doing well - it generates 4 different files with the results of each test suit.

Snow
  • 31
  • 1
  • 4
  • I found two new things to me, but they don't work as I expect it to be. Here in Mochawesome we can see that it's recommended to use '--require mochawesome/register' flag if you run your tests in --parallel mode. Also here, in CodeceptJS` documentation you can add that kind of flags in your configuration file https://codecept.io/configuration/#require . I tried this out but the result is the same - report contains only the results of the last worker – Snow Jan 13 '22 at 07:41
  • I was able to successfully generate Allure reports for parallel tests in a proof of concept. I used the Selenoid option, and the Allure reports were able to aggregate all results from parallel runners. With mocha multi reporter / mochawesome, I wasn't able to get it to work. Did you look into that? – Shan Plourde Jul 15 '22 at 03:43

0 Answers0