How to run kotest which are not tagged by default?

Question

In the kotest framework, there is a way to group tests with custom tags and you can run the particular group by selecting via Gradle parameter like gradle test -Dkotest.tags="TestGroupOne"

I have two test cases one is with a tag and another one is without a tag

object Linux : Tag()

class MyTests : StringSpec({
    "without tag" {
        "hello".length shouldBe 5
    }
    "with tag".config(tags = setOf(Linux)) {
        "world" should startWith("wo2")
    }
})

Now if I run gradle build it runs both tests, but I would like to run the tests which are not tagged by default. In the above example, the test without tag should run if there is no parameter passed in gradle

One way to achieve this behaviour is by adding a task in build.gradle.kts file

val test by tasks.getting(Test::class) {
    systemProperties = System.getProperties()
        .toList()
        .associate { it.first.toString() to it.second }
    if(!systemProperties.containsKey("kotest.tags"))
        systemProperties["kotest.tags"] = "!Linux"
}

As you can see, when there is no parameter passed for -Dkotest.tags I'm manually adding the value !Linux to the systemProperties so that the build script will run tests which are not tagged by default.

Question: Is there any better way to achieve this?

I even tried adding systemProp.gradle.kotest.tags="!Linux" in gradle.properties file but there is no effect.

Answer 1

Your solution is not very robust in the sense that you depend on the concrete tag that is used. It seems that there is no easier solution for that, because the syntax for tag expressions does not allow to write something like "!any".

However, it is possible to write a Kotest extension for what you need that looks like this:

import io.kotest.core.TagExpression
import io.kotest.core.config.ProjectConfiguration
import io.kotest.core.extensions.ProjectExtension
import io.kotest.core.extensions.TestCaseExtension
import io.kotest.core.project.ProjectContext
import io.kotest.core.test.TestCase
import io.kotest.core.test.TestResult
import io.kotest.engine.tags.runtimeTags

object NoTagsExtension : TestCaseExtension, ProjectExtension {
    private lateinit var config: ProjectConfiguration

    override suspend fun interceptProject(context: ProjectContext, callback: suspend (ProjectContext) -> Unit) {
        config = context.configuration
        callback(context)
    }

    override suspend fun intercept(testCase: TestCase, execute: suspend (TestCase) -> TestResult): TestResult {
        return if (config.runtimeTags().expression == TagExpression.Empty.expression) {
            if (testCase.spec.tags().isEmpty() && testCase.config.tags.isEmpty()) {
                execute(testCase)
            } else TestResult.Ignored("currently running only tests without tags")
        } else execute(testCase)
    }
}

The first function interceptProject is just there to obtain the project configuration in order to determine the specified set of tags for the current test run.

The second function intercept is for each test-case. There we determine if any tags have been specified. If no tags were specified (i.e. we have an empty tag expression), we skip all test where any tag has been configured at the spec or test-case. Otherwise, we execute the test normally, and it will then possibly ignored by Kotlin's built-in mechanisms, depending on its tags.

The extension can be activated project-wide in the ProjectConfig:

class ProjectConfig : AbstractProjectConfig() {
    override fun extensions(): List<Extension> = super.extensions() + NoTagsExtension
}

Now, with the extension in place, only tests without tag run by default, regardless of what tags you use in your project.

Answer 2

You can set default tags in your gradle configuration. For example:

tasks.withType<Test> {
  useJUnitPlatform()
  val tags = System.getProperty("kotest.tags")
  if (tags != null) systemProperties.put("kotest.tags", tags)
  else systemProperties.put("kotest.tags", "YourDefaultTags")
}