1

We are running "sbt test" to execute our tests on multiple projects. But we now have tests which need to execute in a specific order. So first all tests with @attributeX and after all these tests are completed we want to execute every test which doesn't contain attribute @attributeX.

sbt testOnly -- -n attributeX
sbt testOnly -- -l attributeX

This implementation works fine, but our issue is with the test reports. When the first run is completed we have our test reports for these tests, but when the second run has completed, the XML files are all generated again and this deleted the results of the first run.

Is there a easy way to configure in either build.sbt that we want to run groups of tests in a specific order (so we still want to run tests parallel but only within the group). Or can we somehow merge the two test results into one when executing these commands separately?

Josjr87
  • 193
  • 1
  • 13

1 Answers1

0

You could order them on per-suite level, for example:

@DoNotDiscover
class MyTest1 extends AnyFlatSpec ...

@DoNotDiscover
class MyTest2 extends AnyFlatSpec ...

and then:

import org.scalatest.Sequential

class SequentialSuite extends Sequential(
   new MyTest1,
   new MyTest2
)

Now when you run sbt test it will first execute tests in MyTest1, and then in MyTest2.

That's regardless of your tags.
Which kinda makes sense, because tags are usually fast/slow/uses-db.
They are usually used for local-only/environment-dependent etc.

insan-e
  • 3,883
  • 3
  • 18
  • 43
  • Hi, thanks for your help. We indeed need it to split the tests which require production data and which do not require production data for testing. We restore the database and afterwards we first execute all tests which do need production data and afterwards we are truncating the entire database and execute everything that doesn't need data. But this to me looks like double administration right? Is here no other solution inside SBT? – Josjr87 Apr 15 '21 at 10:26
  • Nothing as simple as this that I know of.. :D – insan-e Apr 15 '21 at 11:09