How to get intellij to offer a text diff comparison on failed tests

Question

I'm writing a Scala test with some ScalaTest matchers.

When my test fails, intellij says something like

{"count":3,"pagination":{"offset":0,"limit":100},"content":{"uri":"http://locahost.com/catalogue/content?order=Query&id=18,20,19"},"list":[{"id":"18","position":27},{"id":"20","position":341},{"id":"19","position":33}]} was not equal to {"count":3,"pagination":{"offset":0,"limit":100},"content":{"uri":"http://locahost.com/catalogue/content?order=Query&id=18,20,19"},"list":[{"id":"18","timestamp":"2015-01-28T11:55:44.494Z","content":"Episode","position":27},{"id":"20","timestamp":"2015-01-19T11:55:44.494Z","content":"Program","position":341},{"id":"19","timestamp":"2015-01-17T11:55:44.494Z","content":"Episode","position":33}]}
org.scalatest.exceptions.TestFailedException: {"count":3,"pagination":{"offset":0,"limit":100},"content":{"uri":"http://locahost.com/catalogue/content?order=Query&id=18,20,19"},"list":[{"id":"18","position":27},{"id":"20","position":341},{"id":"19","position":33}]} was not equal to {"count":3,"pagination":{"offset":0,"limit":100},"content":{"uri":"http://locahost.com/catalogue/content?order=Query&id=18,20,19"},"list":[{"id":"18","timestamp":"2015-01-28T11:55:44.494Z","content":"Episode","position":27},{"id":"20","timestamp":"2015-01-19T11:55:44.494Z","content":"Program","position":341},{"id":"19","timestamp":"2015-01-17T11:55:44.494Z","content":"Episode","position":33}]}
at    org.scalatest.MatchersHelper$.newTestFailedException(MatchersHelper.scala:160)
at org.scalatest.Matchers$ShouldMethodHelper$.shouldMatcher(Matchers.scala:6231)
at org.scalatest.Matchers$AnyShouldWrapper.should(Matchers.scala:6265)
...

However, intellij does not give me the handy see difference in the text feature.

I thought it might be because I'm comparing 2 objects

  val responseBody = responseAs[JsValue]
  responseBody should be(viewingByAccountIdResponseJson)

but changing it to

assert(responseBody.toString() === viewingByAccountIdResponseJson.toString())

Does not allow me to do a text comparison either.

Is there any way to configure intellij to do this?

(I'm currently using a FlatSpec with Matchers)

Note: This is related to this question Formatting output so that Intellij Idea shows diffs for two texts

However, even using the syntax recommended that intellij might pick up, it does not work.

Using Idea 14.0.2, ScalaTest 2.2.1 and Scala 2.11.4 if I compare two strings `x` and `y` using `assert(x === y)` or `x should be (y)` I see `"a[bc]de" did not equal "a[BC]de"` which I think is the behavior you want. Are you sure you don't have some special configuration that is getting in your way? Or maybe you're just using an old version of ScalaTest? I believe in any case it only works for strings. — Spiro Michaylov, Jan 30 '15 at 15:42
Does it give you the link which compares the 2 text highlighting differences? I've found with java+groovy it does that, just not in my Scala Tests. It means comparing json strings, is quite a challenge to work out what is different. — Bruce Lowe, Jan 31 '15 at 15:13
No, it doesn't. Do you have any reason to believe ScalaTest does this at all, outside the context of IntelliJ? — Spiro Michaylov, Feb 01 '15 at 19:53

score 5 · Answer 1 · answered Mar 06 '15 at 10:42

5

I see this is not possible at the moment and as of the current moment in time, is a feature request:

https://youtrack.jetbrains.com/issue/SCL-4867

answered Mar 06 '15 at 10:42

Bruce Lowe

6,063
1
36
47

The problem here is that by the time IntelliJ sees the output the full expected text and the full actual text have been stripped away. – Felix Leipold Mar 17 '20 at 12:48

Sergey Romanovsky · Answer 2 · 2019-09-17T23:52:42.330

As intellij feature request mentioned by Bruce celebrates it's 7th birthday, some of us have lost their hope (still don't forget to +1 it). Here's an ugly script which allows you to mitigate the issue a bit. Just copy This was not equal to That line and feed it to the stdin of this script:

| scalatest-diff

cat > ~/bin/scalatest-diff
#!/usr/bin/perl

my $in = join("", <STDIN>);
if( $in =~ m/[^:]+\: (.+?) was not equal to (.+)/so ) {
    my $this = $1;
    my $that = $2;
    $this =~ s/,/,\n/g;
    $that =~ s/,/,\n/g;
    open(thisFile, ">", "/tmp/this");
    open(thatFile, ">", "/tmp/that");
    print thisFile $this; close thisFile;
    print thatFile $that; close thatFile;
    exec("vimdiff /tmp/this /tmp/that");
}
<Ctrl-D>
chmod a+x ~/bin/scalatest-diff

P.S. feel free to change vimdiff to your favorite differ.

score 1 · Answer 3 · edited Mar 17 '20 at 14:14

The problem is that ScalaTest is trying to nicely format the left and the right object before displaying the difference.

You can get around this by defining your own matcher that throws a less fancy TestFailedException. Here is a sketch of how to achieve this in a matcher-based test:


  it must "show diff" in {
    """Hello
      |World!
      |""".stripMargin shouldEqualPlainly
      ("""Hello
         |ScalaTest.
         |""".stripMargin)
  }

  implicit class PlainEquality[T](leftSideValue: T) {
    // Like should equal, but does not try to mark diffs in strings with square brackets,
    // so that IntelliJ can show a proper diff.
    def shouldEqualPlainly(right: Any)(implicit equality: Equality[T]): Assertion =
      if (!equality.areEqual(leftSideValue, right)) {
        throw new TestFailedException(
          (e: StackDepthException) => Some(s"""${leftSideValue} did not equal ${right}"""),
          None,
          Position.here
        )
      } else Succeeded
  }

This will let you click and give you something like this:

How to get intellij to offer a text diff comparison on failed tests

3 Answers3