1

I'm running a caliper benchmark with multiple time* methods and two parameters and at the end get a nice table like

parameter1 parameter2 benchmark

After a small change to a method body I get another nice table like

parameter2 benchmark parameter1

This keeps my emacs skills refreshed, but I'd prefer to get a fixed column order. Is it possible?

A side question: How does the order get actually determined? IMHO, using the declaration would be best as this allows to get the desired order by a trivial code change.

maaartinus
  • 44,714
  • 32
  • 161
  • 320

1 Answers1

0

The fact that you're concerned about the column ordering suggests that you're using it to interpret the results of a given run. That's generally discouraged.

For each run, several measurements are taken (which are themselves average runtime per rep), but only the mean of the measurements are reported. It gives you no indication of the spread of the data, the existence of outliers, etc. In the current version of the Caliper webapp, the boxplots on the end of the bars help to show a bit of this information and in the upcoming release full summary statistics will be available.

gk5885
  • 3,742
  • 21
  • 16
  • In my use case I don't care about the spread: The sigma is about 1% of the value and the differences between versions are tens or hundreds of percent. Moreover, the timings are fairly reproducible. I think I need to have a look at the webapp anyway. – maaartinus Dec 06 '12 at 21:58
  • 1
    If the output isn't for interpreting, what *is* it for? I'm having the same issue, it was sorting the way I wanted for a while, but now it has flipped the order. – BeeOnRope Dec 12 '12 at 07:05
  • The console output is designed to give basic status on the progression of a run. Each run contains a number of trials which contain a number of measurements which represent a number of reps. There really is quite a bit of data and we haven't found a good way to distill it down to something that fits on the console without losing too much valuable information. We don't trust it. No performance decisions for Guava are ever made based on the console output. – gk5885 Dec 12 '12 at 09:33