JMH Benchmarking - Concise way to compare runtime of alternate implementations

Question

I have two implementations of the same custom class:

private List<String> a = Util.myCustomClass();
private List<String> b = Util2.myCustomClass();

and would like to compare the runtime of each of their functions (which all have the same name). Currently, my benchmark tests look like:

@Benchmark
public boolean contains_val_a() {
   return a.contains(val);
}

@Benchmark
public boolean contains_val_b() {
  return b.contains(val);
}

And I repeat this parallel structure for 25 or so different functions (writing each function twice because of the two implementations). Is there a way for me to only write the 25 @Benchmark functions and have jmh run each function for both implementations?

score 2 · Accepted Answer · answered Mar 02 '15 at 09:11

You could use @Param to define the (stringly) the classes you want to load and have it initialize the class under test in a @Setup method, as explained in this sample: https://github.com/ktoso/sbt-jmh/blob/master/src/sbt-test/sbt-jmh/jmh-run/src/main/scala/org/openjdk/jmh/samples/JMHSample_27_Params.scala

In essence

@Param(Array("a", "b"))
val name: String = ""

var thing: CommonInterface = ""

@Setup
def setup(): Unit = name match {
  case "a" => new A
  case "b" => new B
}

JMH will then include a "param" column in the test results, so you know which result was for which value.

JMH Benchmarking - Concise way to compare runtime of alternate implementations

1 Answers1