1

I am trying to benchmark Intel TBB bounded concurrent queue enqueue using Google benchmark. What I understand is that Google benchmark runs for n number of iterations till the latency becomes stable and returns the average timings. I want to find percentile latency (99, 90, 50, 10). I searched but couldn't find anything that can help me do this. The closest thing I found was using a custom statistics function provided by the user. But that only works with repetitions (running iterations for n repetitions) and works on the average timings returned, i.e, if I run the test for n number of repetitions, average timings are returned for n repetitions and I can find percentiles from the average timings. I don't get all the latencies recorded in all the iterations. I get only the average timings for each repetition to find percentile.

#include<benchmark.h>
#include<concurrent_queue.h>
#include<concurrent_priority_queue.h>
#include<iostream>
#include<string.h>
#include<numeric>
#include<cmath.h>

const auto PercentileCalc = [] (const std::vector<double>& v) {
    std::vector<double> delta(v.begin(), v.end());
    std::sort(delta.begin(), delta.end());
    return delta[std::ceil(99L*(delta.size()/100))];
};

static void benchmarkTBB(benchmark::State& state) {
    char* src = new char(0);
    tbb::concurrent_bounded_queue<char*> m_queue;
    m_queue.set_capacity(500000);
    for(auto _:state) {
        m_queue.try_push(src);
    }
}
BENCHMARK(benchmarkTBB)->Repititions(100)->ComputeStatistics("percentile", PercentileCalc);
BENCHMARK_MAIN();

Is there any way to calculate percentile latencies using google Benchmark ?

Bishal Mazumdar
  • 11
  • 5
  • the idea is that the individual iterations aren't that interesting (and are potentially quite noisy) so the only useful information is the average latency of all the iterations. assuming that you believe the average latency of iterations is ~the latency, then repeating that test and getting statistics on _that_ is the interesting thing. – dma Sep 07 '20 at 12:13
  • @dma Thanks for the reply. But in the low latency world, we are concerned out the outliers and the noise. We need to know (for outliers) if they happen and why they happen and check if anything can be done to get rid of it. Percentiles can be useful there. – Bishal Mazumdar Sep 11 '20 at 06:45

0 Answers0