2

As far as I know TestCafe default behaviour is to run tests in parallel. Indeed the browsers function accepts an array of browser (which is cool). What I would like to do however is quite different. I have fixtures based on area of my portal (search, payment etc...) and so I'd like to know if it's possible to run these tests in CLI in parallel as they are orthogonal. The scope is of course to improve the execution time as the number test cases will grow. On the other hand I'd like also to catch the failures meaning that if a test ran in parallel on a specific metadata filter fails possibly we would like to stop the others too.

I am not using TestCafe's docker but our custom one with just Firefox, Chrome installed and we launch of tests in headless mode.

As a last point a great thing would be if we could run these scenario/metadata in parallel but somehow at the end of the test suite gather the reports together.

I understand the question is not easy especially because it involves either TestCafe or GitlabCi but probably someone else faced this problem too.

Thank you

Aurélien
  • 33
  • 5

1 Answers1

2

If I understand you correctly, the behavior you described can be achieved by dividing the test execution among multiple CI jobs. For example, each CI job can test a particular area of your portal. For that, run TestCafe with specified metadata of your fixture/test. Also, most of the CI systems allow you to cancel all other jobs in a pipeline if one of the jobs fails (unfortunately, Gitlab hasn't released this feature yet).

On the other hand, you can use TestCafe's programmatic API: create multiple TestCafe runners, each running the desired subset of tests. However, at the end of the test execution, you'll need to merge generated reports into one report manually. Check this answer to get an idea of how to create multiple runners.

wentwrong
  • 711
  • 4
  • 7