I have a suite of system tests that are driven by XUnit (2.x). These run against a freshly deployed set of microservices.
(I know many feel that tests shouldn't have an order dependency and should be independent, but what we have works well as a complete suite to run after every deployment).
We need an ordering, as there is some initial tests that act as tests and as setup steps against the services. These are necessary before the other tests can run. One thing I could do is move the setup steps into an Assembly Fixture. However, I do value them being in test-cases themselves.
We are using Xunit.extensions.ordering to enforce the order, but the tests currently run sequentially.
I'd like to introduce parallelism for the later tests, like the following:-
Non-Parallel tests (initial setup)
- TestClass 1 [Order(1)]
- TestClass 2 [Order(2)]
- TestClass 3 [Order(3)]
Then, in Parallel
[Order(4)]
- TestClass4, TestClass5, TestClass6 ....
My first attempt at enabling parallelisation in XUnit appears to show that the order is no longer respected (by removing the CollectionBehavior(DisableTestParallelization = true) assembly attribute). This appears to disable any ordering.
I can see that there is a way to disable parallel running for a specific collection, using [CollectionDefinition(DisableParallelization = true)]
However, the XUnit docs say that this results in parallel-enabled tests running first, then sequential ones after. I'd need the opposite of this.
Is this possible to do?
