-1

I would like to define an abstract base class that defines generic reusable test cases that I want to implement over and over again when testing various APIs. The idea is to define the tests once, and then I can just inherit from the generic test in order to get the test definitions, and all my test code has to do is implement the specifics of how to execute the tests. That way I never miss good test cases, I am not having to re-invent the wheel, nor making changes in dozens of places when I think up another good test case to test.

In essence, I'm doing parameterized testing, where the parameters are defined in the base class. The expected result is also defined in the base class. However, there are times when the expected result needs to be overridden. For example, an ordinary string field might allow any characters, whereas an object name field may (or in my case DOES) limit which characters are legal. For example, a String field allows "?" in the value, a Name field does not.

I'm trying to figure out a clean way of defining the parameter ONCE, in the base class, and defining the expected result only once but with the option of overriding it in the implementing class where the context requires it.

The problem I have is that when you override a method, you have to replace its implementation. You can't just replace a portion of it. I want to replace the expected outcome without overriding the parameter portion. Nor do I consider it a clean or desirable solution to break every test into two methods, one that provides the parameter and one that defines the expected behavior.

One option I am considering is using annotations to define the expected results. Then I can override the annotation, and then delegate the implementation to the base class implementation. As long as the base class uses the annotation to decide how to behave, it should work.

For example: (in pseudo-code, not real Java)

abstract class foo {

  @fubar(true)
  void test1() { doSomething( param1 ) }

  @fubar(true)
  void test2() { doSomething( param2 ) }

  @fubar(false)
  void test3() { doSomething( param3 ) }

  boolean isFubar( void ) { ... }  // generic annotation grabber

  void doSomething( p ) {
    if ( isFubar() ) 
      doThis( p ) 
    else 
      doThat( p );
  }

  // abstract methods to be defined in the implementing class
  void doThis( p );
  void doThat( p );
}

class bar extends foo {

  // change the expected result for test2 only
  @fubar(false)
  void test2() { super.test2(); }

  // define how to execute the tests
  void doThis(p) { ... }
  void doThat(p) { ... }

}

class baz extends bar {

  // only changes the semantics of test3, nothing else
  @fubar(true)
  void test3() { super.test3() }

}

Given this hierarchy, foo.test1(), bar.test1(), and baz.test1() all do exactly the same thing. Whereas foo.test2() does one thing, and bar.test2() and baz.test2() does something else. Similarly, foo.test3() and bar.test3() do one thing, but baz.test3() will be different.

Can I use annotations to accomplish this behavior? If so, what would isFubar look like? I've yet to see an example of this kind of reflection where the method name is not known.

As a side note, if there is a cleaner way to accomplish the intended behavior, I'd be happy to hear what it is.

John Arrowwood
  • 2,370
  • 2
  • 21
  • 32
  • 1
    Smells like a possible XY Problem. You might want to look into AOP, or try to find a more object-oriented approach. – shmosel Jul 26 '17 at 00:35
  • 1
    Please elaborate on *what* you're trying to accomplish rather than *how*. Going by your snippet, it would make much more sense to simply override `isFubar()` to return true or false, and discard the annotation entirely. – shmosel Jul 26 '17 at 00:42
  • 1
    Also, your question as posed comes across as a code request, which is frowned on around here. Have you made any attempt to to solve the problem? Where are you stuck? – shmosel Jul 26 '17 at 00:45
  • 1
    If you want your base class to provide a value, and you want subclasses to optionally override that value, and you want that value available to a base class method, you simply declare a `protected` method returning the value you want, and make the base class return it's value, and subclasses can override that method and return something else, if they want. Jumping through hoops to try to mimic that very common behavior using annotations is just ... wrong. – Andreas Jul 26 '17 at 00:45
  • 1
    Methods `doThis` and `doThat` in class `foo` must be declared `abstract`. In Java, you don't declared parameters as `void`. Java naming convention is for class names to start with uppercase letter. – Andreas Jul 26 '17 at 00:49
  • @Andreas - I program in a lot of different languages, and when writing pseudo-code I sometimes mix them together. But you got the point, obviously. – John Arrowwood Jul 26 '17 at 12:22
  • The context for this question is in integration testing. I want to define my related test cases in a generic base class. The expected results are also defined in the generic base class. However, sometimes I will need to override the expected results. I want to find a way of doing that WITHOUT having to completely replace the body of the test, or duplicating logic or the test parameters in the instance class. – John Arrowwood Jul 26 '17 at 12:25
  • Example: have a class that defines tests for a generic String field in a database record. It will have 50-100 test cases defined. Most should pass, a few might be expected to fail (e.g. can't use that value). Then there is the record Name field, which has some restrictions. It will inherit from the generic String Field, and should override only the expected outcome for the test cases, leaving the implementation alone. – John Arrowwood Jul 26 '17 at 12:29
  • That is, those few that pass in a normal String field but should be rejected from a name field will have those tests overridden. This is why @shmosel I can't just override isFubar(). – John Arrowwood Jul 26 '17 at 12:31
  • Annotations may not be the ideal solution to this problem. But none of the sample code I've seen seems to work where you don't know the specific place where the annotation is applied. Which is the reason I'm asking: CAN IT WORK AT ALL IN THIS SITUATION? – John Arrowwood Jul 26 '17 at 12:33
  • And if there is a better way to achieve what I'm trying to do, I'm all ears. – John Arrowwood Jul 26 '17 at 12:37
  • I updated the question to be more explicit about what I'm after. – John Arrowwood Jul 26 '17 at 13:11
  • If I understand you right, you have a base class `X` and write 100+ test cases for it in a `XTest` class. You then have a subclass `Y` of `X`, and all the test cases are the same, except the expected output is different in a few of them. What is wrong with just creating subclass `YTest` of `XTest` and override the construction logic to create `Y` instead of `X`, and overriding the few test cases that have different outcome? – Andreas Jul 26 '17 at 14:05
  • And if you don't want to override the entire test case method, but just the expected output, then you split that test case method in two, so you can override just the expected output, without overriding the invocation logic. – Andreas Jul 26 '17 at 14:07
  • No, I have a base TEST class, X. Then I define another more specific TEST class, Y, which extends X and overrides only some of its behavior. Along with the rest of the letters of the alphabet, of course. Then when testing some API endpoint, I will define test classes that extend either X or Y as appropriate, and implement the key parts that make the tests work. The goal is to strictly adhere to the DRY principle, in the face of the extreme repetition of testing dozens of API endpoints, all of which merit thorough testing. – John Arrowwood Jul 26 '17 at 16:22
  • You should [edit] your question to include all those clarifications. They go mostly unnoticed down here in the comments ("below the fold", no less). – Kevin J. Chase Jul 26 '17 at 19:52
  • I did, but if it is still not clear, suggestions for further edits are welcome. – John Arrowwood Jul 26 '17 at 20:07

1 Answers1

0

To find the earliest matching annotation in a call stack, you could use a function like this:

static <T extends Annotation> T seekAnnotation(Class<T> annotationClass) {
    T annotation = null;
    try {
        for (StackTraceElement ste : Thread.currentThread().getStackTrace()) {
            T found = seekAnnotation(annotationClass, ste);
            if (found != null) {
                annotation = found;
            }
        }
    } catch (Exception e) {
        //
    }
    return annotation;
}

static <T extends Annotation> T seekAnnotation(Class<T> annotationClass, StackTraceElement ste) {
    T annotation = null;
    try {
        Class<?> claz = Class.forName(ste.getClassName());
        Method method = claz.getDeclaredMethod(ste.getMethodName());
        annotation = method.getAnnotation(annotationClass);
    } catch (Exception e) {
        //
    }
    return annotation;
}

There is more work to make it robust, but this would be the basic idea. So your method isFubar() would look like

boolean isFubar() {
    return seekAnnotation(fubar.class).value();
}
Evan Jones
  • 876
  • 4
  • 9
  • Unfortunately, in this example things are oversimplified. The definition of `isFubar` needs to work regardless of which method has the annotation. Because there may be 50-100 methods in the class which have the annotation. This is why I mentioned traversing the call stack. – John Arrowwood Jul 26 '17 at 12:36
  • Here is a second version that walks the stack trace. – Evan Jones Jul 27 '17 at 01:53
  • Nice! It looks like, unless it errors out while looping, that it will take the value of the annotation that is highest in the call stack, which is exactly what I was wanting. And it is able to do it without having to know the specific method name, which is what I was unclear on how to do. I am going to mark this as the answer to my question. – John Arrowwood Jul 27 '17 at 13:25
  • I am curious, though: what needs to be added to make it more robust? (Not asking for code, just wondering what you had in mind) – John Arrowwood Jul 27 '17 at 13:26
  • One issue is that if your method signatures have parameters then you will have to do more work to find the method on the class. The Class.getDeclaredMethod method also takes a list of parameter classes, so my code will never hit a parameterized method, You can use getDeclaredMethods() but then if you get multiple results you will not know which method was actually called. I don't have other concrete example in my head, but I have suspicions of other problems. Depending on you use case these may be irrelevant. – Evan Jones Jul 28 '17 at 00:03