Elixir: Test failure messages

Question

Using Elixir's ExUnit for unit testing, everyone seems excited about the test failure messages. However, if I am testing anything other than ==, I can't seem to get a useful error message.

For example

assert String.contains(my_string, "expected string")

    Expected truthy, got false

I prefer to write my tests so that my failures are more informative, allowing me to quickly iterate on the feedback they give. Is there a way to get a more informative test failure? e.g.

Expected "an unexpected string" to contain "expected string"

You could write an `assert_contains` macro which does this or pass a custom message to `assert` which it'll print on failure, but I'm not sure what kind of solution exactly you're looking for. — Dogbert, Aug 20 '16 at 01:38
All asset does is check truthiness. You can supply a message, but on its own, it won't introspect what you're running that produces the result under assertion. This has come up before, e.g., https://groups.google.com/forum/m/#!topic/elixir-lang-core/KYAkagiKr0w, but that's still not automatic like you're asking for. — Dave Newton, Aug 20 '16 at 01:39

score 10 · Accepted Answer · answered Aug 20 '16 at 20:52

For the particular case of testing against String.contains?, you could consider the regex match operator =~, as it will report a better test failure message out of the box.

Generally though, ExUnit's assert won't know about the inner semantics of every function in the standard library. This is why you can provide a custom error message:

assert String.contains?(my_string, "expected string"),
       ~s(Expected #{inspect my_string} to contain "expected string")

If you have many asserts for String.contains?/2, you can define your own assertion:

def assert_contains(string, substring) do
  assert String.contains?(string, substring),
         "Expected #{inspect string} to contain #{inspect substring}"
end

test "my string" do
  assert_contains my_string, "expected string"
end

Here are the expressions assert and refute will recognise and tailor the output for in Elixir 1.3:

= match operator
match?/2 function
! negation
== equality operator
< less than
> greater than
<= less than or equal
>= greater than or equal
=== strict equality (distinguishes floats and integers)
=~ regular expression match operator
!== strict inequality
!= inequality
in membership in enumerable

Most of these get the same special treatment: they are infix operators, and assert will report on the lhs, rhs values it sees.

Elixir: Test failure messages

1 Answers1