How to create custom JUnit4 assertions that don't show in the Failure Trace

Question

I'd like to add some custom assertions to our code base that properly hide from the failure trace. I know how to write a public static method that someone can statically import. I know how to reuse old assertions or throw a new AssertionError.

What I can't figure out how to do is keep the new custom assertions out of the Failure Trace. We're used to the first hit in the failure trace NOT being the assertion code itself but the test code that called the assertion.

I know there is a filtertrace attribute that controls filtering the stack but I can't find any good documentation of what I'd have to do to add the new assertions to the filter.

An example of what I want to do:

package testassertions;

import static newassertions.MyAssertions.myAssertTrue;

import org.junit.Test;

public class ExampleTest {
    @Test
    public void myAssertTruePassing() { myAssertTrue(true); }

    @Test
    public void myAssertTrueFailing() { myAssertTrue(false); }
}

package newassertions;

import static org.junit.Assert.assertTrue;

public class MyAssertions {

    public static void myAssertTrue(boolean b) {
        assertTrue(b);
    }
}

Failure Trace of myAssertTrueFailing() shows:

java.lang.AssertionError
    at newassertions.MyAssertions.myAssertTrue(MyAssertions.java:8)
    at testassertions.ExampleTest.myAssertTrueFailing(ExampleTest.java:12)

I need it to only show:

java.lang.AssertionError
    at testassertions.ExampleTest.myAssertTrueFailing(ExampleTest.java:12)

score 5 · Accepted Answer · edited May 23 '17 at 12:22

As mentioned in another question about cleaning noise from stack traces, filtering classes from within your IDE is probably the easiest solution. In fact, the stack traces you've shown in your question are already filtered.

If you really wanted to do this in code, you could add filtering to your custom assertion class something like below:

package newassertions;

import static org.junit.Assert.assertTrue;
import java.util.ArrayList;

public class MyAssertions {

    public static void myAssertTrue(boolean b) {
        try {
            assertTrue(b);
        } catch (AssertionError e) {
            filterStackTrace(e);
            throw e;
        }
    }

    private static void filterStackTrace(AssertionError error) {
        StackTraceElement[] stackTrace = error.getStackTrace();
        if (null != stackTrace) {
            ArrayList<StackTraceElement> filteredStackTrace = new ArrayList<StackTraceElement>();
            for (StackTraceElement e : stackTrace) {
                if (!"newassertions.MyAssertions".equals(e.getClassName())) {
                    filteredStackTrace.add(e);
                }
            }
            error.setStackTrace(filteredStackTrace.toArray(new StackTraceElement[0]));
        }
    }
}

The name of the enclosing class 'newassertions.MyAssertions' (hard-coded) is filtered from the stack trace in this example. This mechanism would obviously also work to filter the stack trace from an AssertionError that you create yourself and not just those raised from other assertions.

This is missing some semicolons at the top but looks very promising. — candied_orange, Jul 21 '15 at 13:38
Re. edit comment: Darn those fat fingered gremlins. Anyway, this looks good. I'm tempted to add some configuration code that would disable the filter incase any purest needs to see the real stack trace but would leave this code solution the default. That is, unless I get authorized to change our configuration so I can filter one of our packages. Just need a chance to code this up at work to test it in our environment. — candied_orange, Jul 22 '15 at 01:47

score 4 · Answer 2 · answered Jul 21 '15 at 09:03

Have you considered using org.junit.Assert.assertThat with Hamcrest matchers?

With Hamcrest, you wouldn't need to change the assertion methods, but instead implement your own matchers. For example, to verify a BCrypt-hashed password matches the plain password, write a matcher like this:

public class MatchesPassword extends TypeSafeMatcher<String> {

    private static final PasswordEncoder PASSWORD_ENCODER = new BCryptPasswordEncoder();

    private final String password;

    public MatchesPassword(String password) {
        this.password = password;
    }

    @Override
    protected boolean matchesSafely(String encodedPassword) {
        return PASSWORD_ENCODER.matches(password, encodedPassword);
    }

    @Override
    public void describeTo(Description description) {
        description.appendText("matches password ");
        description.appendValue(password);
    }
}

Next, add a method somewhere that you can statically import:

public class CustomMatchers {

    public static Matcher<String> matchesPassword(String password) {
        return new MatchesPassword(password);
    }

}

Finally, write your test like this:

@Test
public void passwordShouldMatch() {
    PasswordEncoder passwordEncoder = new BCryptPasswordEncoder()
    String plainPassword = "secret";
    String hashedPassword = passwordEncoder.encode(plainPassword);

    assertThat(hashedPassword, matchesPassword(plainPassword));
}

A mismatch will be logged to the console like this:

java.lang.AssertionError: 
Expected: matches password "wrong"
     but: was "$2a$10$5lOyLzUeKMAYPJ5A3y5KfOi747DocksLPHgR7GG3XD8pjp8mhaf0m"
    at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
    at org.junit.Assert.assertThat(Assert.java:956)
    at org.junit.Assert.assertThat(Assert.java:923)
    ...

Note: BCryptPasswordEncoder is from Spring Security and just used as an example.

We don't use assertThat, matchers, Hamcrest, or Spring. We're on java 5. Yeah I know. It's what I deal with. — candied_orange, Jul 21 '15 at 11:46
Which version of JUnit are you using? AFAIK, JUnit added `assertThat` in 4.4 (from 2007) and they also include basic matchers from Hamcrest (so you won't need an additional library). Spring was just used as an example here, you won't need it. — hzpz, Jul 21 '15 at 13:14
Did you check which JUnit version you have available and if you can use `assertThat`? — hzpz, Jul 27 '15 at 17:54

Franz Becker · Answer 3 · 2015-07-24T15:22:02.857

My go-with solution would also be an IDE filter as others already suggested. If you do a "hard-coded" solution this will be less traceable in an automated build process.

In Eclipse you can open the preferences and select Java -> JUnit and add classes or packages using the buttons on the right.

But just for the fun of it:

If you really want to do it programmatically @gar's solution sounds quite reasonable. However, if you have a bigger amount of assertions this might be a bit tedious.

What you could also do is to subclass AssertionError and filter the stacktrace at its root.

public class MyAssertionError extends AssertionError {

    public MyAssertionError(String message) {
        super(message);
    }

    @Override
    public synchronized Throwable fillInStackTrace() {
        super.fillInStackTrace();
        filterStackTrace();
        return this;
    }

    protected void filterStackTrace() {
        StackTraceElement[] trace = getStackTrace();
        ArrayList<StackTraceElement> list = new ArrayList<StackTraceElement>(trace.length);
        for (StackTraceElement element : trace) {
            if (!element.getClassName().equals("newassertions.MyAssertions")) {
                list.add(element);
            }
        }
        this.setStackTrace(list.toArray(new StackTraceElement[0]));
    }

}

Note two things here: 1) the class name of a StackTraceElement can never be null so its fine to write the constant on the right side 2) if you put all your assertions in a separate package you could also write element.getClassName().startsWith("newassertions")

Your assertion class would then look like this:

package newassertions;

public class MyAssertions {

    public static void myAssertTrue(boolean b) {
        if (!b) {
            fail(null);
        }
    }

    public static void fail(String message) {
        if (message == null) {
            throw new MyAssertionError(message);
        }
        throw new MyAssertionError(message);
    }


}

This way you could not call the methods from Assert but if you write more complex assertions there are few reasons to do this anyway. However, it would keep your assertion code a bit cleaner compared to wrapping everything in big try-catch blocks.

Stefan Ferstl · Answer 4 · 2015-07-21T10:37:41.553

You can use a custom JUnit method rule together with custom asserts. The custom asserts can work with a subtype of AssertionError. This would even allow you to use Junit assertions and the custom assertions together.

Example

Here is an example that uses a custom MyAssert class that throws MyAssertionErrors in case an assertion failes. The JUnit rule handles MyAssertionError and hides any details of the failure trace.

public class RuleTest {

  @Rule
  public TestVerifier testVerifier = new TestVerifier();

  @Test
  public void myAssertOk() { MyAssert.assertCondition("ok", true); }

  @Test
  public void myAssertNotOk() { MyAssert.assertCondition("nok", false); }

  @Test
  public void junitAssertNotOk() { assertTrue(false); }

  @Test
  public void junitAssertOk() { assertTrue(true); }

  static class TestVerifier implements TestRule {

    @Override
    public Statement apply(Statement base, Description description) {
      return new Statement() {

        @Override
        public void evaluate() throws Throwable {
          try {
            base.evaluate();
          } catch (MyAssertionError t) {
            throw new AssertionError("Test failed: " + description.getMethodName());
          }
        }
      };
    }

  }

  static class MyAssertionError extends AssertionError {
    public MyAssertionError(Object detailMessage) { super(detailMessage); }
  }

  static final class MyAssert {
    public static void assertCondition(String message, boolean condition) {
      if (!condition) { throw new MyAssertionError(message); }
    }
  }
}

Using this custom TestVerifier rule your failure trace will only say:

java.lang.AssertionError: Test failed: verifierTest
    at RuleTest$TestVerifier.apply(RuleTest.java:26)
    at org.junit.rules.RunRules.applyAll(RunRules.java:26)
    ...

In your IDE it will look like this:

IDE screen shot

I don't understand this example. There is nothing custom about assertTrue(). Show me myAssertTrue(). — candied_orange, Jul 21 '15 at 06:01
@CandiedOrange I rephrased my previous answer a bit. In the previous version I missed that you *require* custom assertions. — Stefan Ferstl, Jul 21 '15 at 10:40
This is a bit better but still has a major problem in the failure trace. Due to imports when I run it "public class RuleTest {" is on line 10. When I inspect (click on) the junitNotOK test it correctly reports the problem on line 22 where assertTrue() was called. However, the myAssertNotOK() test reports the problem on line 38. This is not where myAssertTrue() was called, just where the exception was thrown. I need to know were it was called and I need the line that tells me that to be the first line in the failure trace. — candied_orange, Jul 21 '15 at 11:33

How to create custom JUnit4 assertions that don't show in the Failure Trace

4 Answers4