0

I've created a class. Within that class I have several methods that are marked as @Benchmark. I also have a main method that runs the JMH benchmarks:

    System.out.println("NUMBER OF THREADS: "+numOfThreads);
    Options opt = new OptionsBuilder()
            .include(JMHtopToBottom.class.getSimpleName())
            .warmupIterations(5)
            .measurementIterations(3)
            .forks(numOfThreads)
            .build();

    Collection<RunResult> collection = new Runner(opt).run();

My interest is to have:

  1. A setup method that runs only ones - right after the new Runner(opt).run(); and before all of the @Benchmark methods are called (along with their iterations).

  2. As well, to have a tear down method that runs only once right after all the methods runs and before we go back to main.

When I tried @setup and @tear_down (with Level support: Trial/Iteration/Invocation) the methods run several times and not only ones as I wished. Is there a way in JMH to annotate methods so it will run just ones - right after run() and right before the run() is over?

Oliv
  • 10,221
  • 3
  • 55
  • 76
TaliG
  • 191
  • 2
  • 6
  • Hi. I am not aware that this is possible. Maybe it helps if you point out why you need that construction. Others could then tell you alternative approaches or even tell you that you shouldn't do it like that – Matthias Huber Feb 25 '16 at 09:52
  • Hey I need this in order to build a matrix that all of the methods will write to. On the tear down I wish to print that matrix. If the compiler will call the tear down again and again, It'll override the file which I print to write to. – TaliG Feb 25 '16 at 12:51
  • See: http://stackoverflow.com/questions/33790799/jmh-using-the-same-static-object-in-all-benchmark-tests/33808416#33808416 – Aleksey Shipilev Feb 25 '16 at 21:27

1 Answers1

5

You are missing a few things:

  1. Forks are not threads, they are separate processes launched to run each benchmark. I.e if you set forks to 5 any benchmark (in the selected benchmark set) will be run 5 times, each time in a separate VM.
  2. Unless forks=0 (not recommended as benchmark isolation is gone, mixed compilation profiles etc, meant mostly for debugging) all benchmarks are run in separate processes. So each 'Trial' setup/teardown for a given benchmark will run once for that JVM. There is no shared 'Suite' context.

If you absolutely require some 'Suite' level context you'll have to construct it out of VM (e.g. some file that is read on benchmark setup/updated on teardown etc.).

Nitsan Wakart
  • 2,841
  • 22
  • 27