0

I work in a large enterprise using SonarQube 7.9.2. I see a lot of test code using the AssertJ framework incorrectly. I see that SonarSource does have a small set of rules concerning AssertJ, but none of them cover the particular issue I'm looking at.

The following is a sample (one of my test inputs) that shows the pattern I'm looking for:

import static org.assertj.core.api.Assertions.*;

public class AssertThatWithNoPredicate {

    public Foo foo  = new Foo();

    @Test
    public void assertSomething() throws Exception {
        assertThat(foo.getStuff().equals("abc"));
    }

    public static class Foo {
        private String stuff;

        public String getStuff() { return stuff; }
        public void setStuff(String aStuff) { this.stuff = stuff; }
    }
}

The assertion above is defective, as it needs to do this instead:

assertThat(foo.getStuff()).isEqualTo("abc");

So, I set out trying to write a SonarQube java-based rule to check for this. There is documentation for this, but it doesn't go as deep as I would like. It has a few examples, but it doesn't really explain the api in detail. I started with https://github.com/SonarSource/sonar-java/blob/master/docs/CUSTOM_RULES_101.md as my guide.

Following this is what I currently have for the custom rule. It doesn't work properly. I'll go into what fails further down.

package org.sonar.samples.java.checks;

import org.sonar.check.Rule;
import org.sonar.plugins.java.api.JavaFileScanner;
import org.sonar.plugins.java.api.JavaFileScannerContext;
import org.sonar.plugins.java.api.tree.BaseTreeVisitor;
import org.sonar.plugins.java.api.tree.ImportTree;
import org.sonar.plugins.java.api.tree.MemberSelectExpressionTree;
import org.sonar.plugins.java.api.tree.MethodInvocationTree;
import org.sonar.plugins.java.api.tree.Tree.Kind;

@Rule(key = "AssertThatLackingPredicateCheck",
      name = "Call to assertThat method has to have at least one chained predicate method call",
      description = "The AssertJ assertThat method has to have at least one chained predicate method called on the return value, or it does nothing",
      priority = org.sonar.check.Priority.MAJOR,
      tags = {"bug"})
public class AssertThatLackingPredicateCheck extends BaseTreeVisitor implements JavaFileScanner {

    private JavaFileScannerContext context;

    @Override
    public void scanFile(JavaFileScannerContext context) {
        this.context = context;

        scan(context.getTree());
    }

    @Override
    public void visitImport(ImportTree tree) {
        //scan(tree.qualifiedIdentifier());
        System.out.println("ident[" + tree.qualifiedIdentifier() + "]");
        super.visitImport(tree);
    }

    @Override
    public void visitMemberSelectExpression(MemberSelectExpressionTree tree) {
        System.out.println("In visitMemberSelectExpression.");
        scan(tree.annotations());
        System.out.println("tree.expression.firstToken[" + tree.expression().firstToken().text() + "]");
        scan(tree.expression());
        scan(tree.identifier());
        System.out.println("Exiting visitMemberSelectExpression.");
    }

    @Override
    public void visitMethodInvocation(MethodInvocationTree tree) {
        System.out.println("In visitMethodInvocation.");
        System.out.println("methodSelect[" + tree.methodSelect().firstToken().text() +
                           "] kind[" + tree.methodSelect().kind() +
                           "] parent.kind[" + tree.parent().kind() +
                           "]");
        scan(tree.methodSelect());
        scan(tree.typeArguments());
        scan(tree.arguments());
        if (tree.methodSelect().firstToken().text().equals("assertThat") &&
            tree.methodSelect().kind() == Kind.IDENTIFIER &&
            tree.parent().kind() == Kind.EXPRESSION_STATEMENT) {
            System.out.println("Reporting issue.");
            context.reportIssue(this, tree,
                                "Calls to assertThat have to chain predicate method calls, or the assertion does nothing.");
        }
        System.out.println("Exiting visitMethodInvocation.");
    }
}

My test class is this:

package org.sonar.samples.java.checks;

import org.junit.jupiter.api.Test;
import org.sonar.java.checks.verifier.JavaCheckVerifier;

public class AssertThatLackingPredicateCheckTest {
    @Test
    public void assertThatWithNoPredicate() throws Exception {
        JavaCheckVerifier.newVerifier()
            .onFile("src/test/files/AssertThatWithNoPredicate.java")
            .withCheck(new AssertThatLackingPredicateCheck())
            .verifyIssues();
    }

    @Test
    public void assertThatWithValidPredicate() throws Exception {
        JavaCheckVerifier.newVerifier()
            .onFile("src/test/files/AssertThatWithValidPredicate.java")
            .withCheck(new AssertThatLackingPredicateCheck())
            .verifyNoIssues();
    }
}

The first test file is included above. The second one is this:

import static org.assertj.core.api.Assertions.*;

public class AssertThatWithValidPredicate {
    public Foo foo  = new Foo();

    @Test
    public void assertSomething() throws Exception {
        assertThat(foo.getStuff()).isEqualTo("abc");
    }

    public static class Foo {
        private String stuff;

        public String getStuff() { return stuff; }
        public void setStuff(String aStuff) { this.stuff = stuff; }
    }
}

When I run this, the "valid" test actually passes, but considering that I don't really understand the api I'm using, that is very likely a coincidence.

The "invalid" test fails with an odd stacktrace:

java.lang.AssertionError: Unexpected at [10]
    at org.sonar.java.testing.InternalCheckVerifier.assertMultipleIssues(InternalCheckVerifier.java:308)
    at org.sonar.java.testing.InternalCheckVerifier.checkIssues(InternalCheckVerifier.java:232)
    at org.sonar.java.testing.InternalCheckVerifier.verifyAll(InternalCheckVerifier.java:223)
    at org.sonar.java.testing.InternalCheckVerifier.verifyIssues(InternalCheckVerifier.java:168)
    at org.sonar.samples.java.checks.AssertThatLackingPredicateCheckTest.assertThatWithNoPredicate(AssertThatLackingPredicateCheckTest.java:12)

I would also point out that this is not the first time I've tried to solve this problem, but it was several years ago, and addressing tests using an older framework that AssertJ was based on. At that time, I developed an XPath rule for this that actually appeared to work. Unfortunately, I don't have the ability to integrate XPath rules into our SonarQube instance, only Java-based rules can be used.

The old thread is here: http://sonarqube-archive.15.x6.nabble.com/Write-a-custom-XPath-task-that-looks-for-a-method-that-is-NOT-followed-by-a-chained-method-call-td5024017.html . The response does try to walk through the possible Java-based solution, but I didn't quite understand his instructions.

David M. Karr
  • 14,317
  • 20
  • 94
  • 199
  • Why is the rule `https://rules.sonarsource.com/java/RSPEC-2970` not enough? It should check exactly your scenario (`Assertions.assertThat()` without a following assertion) – Thomas Kläger Apr 06 '21 at 19:16
  • Sigh. I searched sonarsource for this, but I assumed it would have an assertj tag. This rule doesn't have that. I will see if I can get this rule added to our profiles. – David M. Karr Apr 06 '21 at 19:31
  • 1
    I've confirmed that this rule does what I need. I hinted to sonarsource that they should amend the tags for this rule. – David M. Karr Apr 06 '21 at 21:13
  • FYI AssertJ uses `@CheckReturnValue` to hint third party checking tools like findbugs that one need to use the returned value. See https://assertj.github.io/doc/#assertj-core-incorrect-usage – Joel Costigliola Apr 07 '21 at 09:50
  • I do notice that my Eclipse Spotbugs plugin does note the same lines that this rule cites, with the somewhat indirect message of "return value must be checked". I understand what that means, but it might take some education to get developers to grok that. What's also interesting is that our SonarQube also runs Findbugs, and I didn't see any issues noted on these lines until I enabled this rule. – David M. Karr Apr 07 '21 at 15:49

0 Answers0