0

I am using benchmark::RegisterBenchmark for benchmarking my code. For simple example I have :

static void BM_Factorial(benchmark::State& state) {
  std::string x = "hello";
  for (auto _ : state) {
    uint64_t fact = 1;
    for(int i=4; i< 100000; i++)
        benchmark::DoNotOptimize(fact *= i);
  }
}

void fun(){
  // Call this multiple times
  benchmark::RegisterBenchmark("timer ", &BM_Factorial);
  // do some work
  benchmark::RunSpecifiedBenchmarks();
}

Init function is doing :

 int dummy = 4;
  std::string arg0 = "--help";
  std::string arg1 = "--benchmark_report_aggregates_only=true";
  std::string arg2 = "--benchmark_display_aggregates_only=true";
  std::string arg3 = "--benchmark_repetitions=10000";
  char *args[4] = {
      const_cast<char *>(arg0.c_str()), const_cast<char *>(arg1.c_str()),
      const_cast<char *>(arg2.c_str()), const_cast<char *>(arg3.c_str())};
  benchmark::Initialize(&dummy, args);

I want to calling my benchmark function multiple times from fun() with some extra arguments. I have replaced benchmark function with factorial for simplicity right now. With this, time reported is always 0. What am I missing here? Another thing is, if i do not write for (auto _ : state) it throws run-time error and exits. The number of iterations reported are always 1000000000.

debonair
  • 2,505
  • 4
  • 33
  • 73
  • 1
    Just *guessing* here. But could it be that your optimizing compiler is being really clever and finding a way to reduce your factorial function to a constant (its result) so that the time it takes at run time *really is* zero.? Have you looked at the generated assembly to see what the compiler *actually* did? – Jesper Juhl Feb 21 '19 at 19:32
  • 1
    You should be using `benchmark::DoNotOptimize()`. –  Feb 21 '19 at 19:48
  • @JesperJuhl that's not an issue, I changed my loop. Anyway it happens for my real function too not just a factorial(). – debonair Feb 21 '19 at 19:48
  • 1
    @debonair The code, as you posted it, is trivially optimizable to a no-op since it has no side-effects. If that's not the case for your example, you need to show us some representative code. –  Feb 21 '19 at 19:49
  • @Frank I tried with `benchmark::DoNotOptimize()` and changing loop, still same effect. – debonair Feb 22 '19 at 00:46
  • 1
    Looks like your current function succeeds at not optimizing away https://godbolt.org/z/OQGeG3, with gcc and clang. – Peter Cordes Feb 22 '19 at 04:27
  • Okay, I found it. So previously I had installed debian package for benchmark library. If I link this, it shows zero and its a debug build. When I explicitly built library from github it worked. Not sure what is wrong with deb package. Thanks all. – debonair Feb 22 '19 at 18:56

0 Answers0