IntelliJ Kotest doesn't show tests failed with exception

Question

Code

I have the following three tests:

import io.kotest.core.spec.style.BehaviorSpec
import io.kotest.matchers.shouldBe

class Example {
    fun blah(number: Int): Int {
        if (number == 1) {
            throw IllegalArgumentException()
        } else {
            return number
        }
    }
}

class ExampleTest : BehaviorSpec({
    Given("example") {
        val example = Example()
        When("calling blah(0)") {
            val result = example.blah(0)
            Then("it returns 0") {
                result shouldBe 0
            }
        }
        When("calling blah(1)") {
            val result = example.blah(1)
            Then("it returns 1") {
                result shouldBe 1
            }
        }
        When("calling blah(2)") {
            val result = example.blah(2)
            Then("it returns 2") {
                result shouldBe 2
            }
        }
    }

})

Problem

The middle test throws an exception which is not expected. I would expect to see 3 tests run, 1 of which failed, but what IntelliJ and Kotest plugin show me is that 2 out of 2 tests passed. I can see in the Test Results side panel that something is not right, but it doesn't have any useful information.

If I navigate to the index.html with tests results, I can see everything correctly. I would like to see the same data in IntelliJ.

Screenshots

IntelliJ output: Note that:

• on the left the test for number 1 is missing
• at the top it says "Tests passed: 2 of 2"
• on the left there are yellow crosses up to "Given", but all tests in "Given" are green

index.html with the test results:

Other information

• Kotest version: 4.6.3
• IntelliJ Kotest plugin version: 1.1.49-IC-2021.2.3

Answer 1

This issue is resolved in 5.1.0.

Answer 2

If an exception is thrown inside the Given or When block, the test initialisation fails. Here is the output if I run that one test only:

It seems that exceptions are only handled inside the Then blocks.

This means that everything that can throw an exception should go into Then blocks, which in turn means that setup and action cannot be shared between the tests:

class ExampleTest : BehaviorSpec({
    Given("example") {
        When("calling blah(0)") {
            Then("it returns 0") {
                val example = Example()
                val result = example.blah(0)
                result shouldBe 0
            }
        }
    }
    
    Given("example") {
        When("calling blah(1)") {
            Then("it returns 1") {
                val example = Example()
                val result = example.blah(1)
                result shouldBe 1
            }
        }
    }

    Given("example") {
        When("calling blah(2)") {
            Then("it returns 2") {
                val example = Example()
                val result = example.blah(2)
                result shouldBe 2
            }
        }
    }

})

This also results in 2 levels of redundant indentation.

An alternative to the above code is to use different kotest testing style, for example ShouldSpec.