Eclipse, Maven & JMH: No matching benchmarks

Question

I have a maven project with the relevant pom.xml as

<dependency>
  <groupId>org.openjdk.jmh</groupId>
  <artifactId>jmh-core</artifactId>
  <version>0.1</version>
</dependency>

I followed the official example here .

My App.java class looks like this:

public class App 
{
    @GenerateMicroBenchmark
    public void run()
   {
        System.out.println("Hello World");
   }

    public static void main( String[] args ) throws RunnerException {
        Options opt = new OptionsBuilder()
        .include(".*" + App.class.getSimpleName() + ".*")
        .forks(1)
        .build();

        new Runner(opt).run();
    }   
}

Still I am getting the following error, when I run it through Eclipse. No matching benchmarks. Miss-spelled regexp? Use -v for verbose output.

score 4 · Accepted Answer · edited May 23 '17 at 11:52

The problem is that JMH looks by default into the directory /META-INF/MicroBenchmarks to look for the benchmarks to run but Maven doesn't compile the classes to this directory. To make it work within Eclipse, please refer to this answer.

However, I would recommend that you run the benchmarks directly on the command-line (Eclipse can add some overhead).

I notice that you are using version 0.1 which is the very first version of JMH. It would be better to update to the last version. The best way to get started with JMH is to generate the project with the archetype jmh-java-benchmark-archetype :

$ mvn archetype:generate \
      -DinteractiveMode=false \
      -DarchetypeGroupId=org.openjdk.jmh \
      -DarchetypeArtifactId=jmh-java-benchmark-archetype \
      -DgroupId=org.sample \
      -DartifactId=jmh-test \
      -Dversion=1.0

This will generate a Maven project from scratch, using the latest JMH version, pre-configured, without you having to worry about the packaging. You can then import this new project into Eclipse as an existing Maven project, and start coding your benchmarks.

After that, you just need to run:

$ cd jmh-test/
$ mvn clean install
$ java -jar target/benchmarks.jar

To get you started quickly, here's a sample benchmark (doing nothing...).

import java.util.concurrent.TimeUnit;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Warmup;

@Warmup(iterations = 5, time = 1000, timeUnit = TimeUnit.MILLISECONDS)
@Measurement(iterations = 10, time = 1000, timeUnit = TimeUnit.MILLISECONDS)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@Fork(3)
@State(Scope.Benchmark)
public class StreamTest {

    @Benchmark
    public void benchmark() {
        // code here
    }

}

Thanks, finally got it to work at my 5th attempt after reading your answer. I tried a lot of different approaches, my 2nd approach was the one you just mentioned. The mistake I was making was that instead of running benchmark.jar I was running the actual MyApp-1.0.jar — AgentX, Nov 08 '15 at 18:47
I have one question, what if I have a couple of method in different/same classes annotated with @Benchmark? — AgentX, Nov 08 '15 at 18:50
@AgentX Each method annotated `@BenchMark` will be executed during the benchmark. So if you want to compare a vs b, you can create 2 annotated methods and they be run. — Tunaki, Nov 08 '15 at 20:16

Eclipse, Maven & JMH: No matching benchmarks

1 Answers1