Get 95th Percentile only through RRD

Question

Yes I know,

"DEF:in=$rrd_file:traffic_in:AVERAGE:step=1",
"DEF:out=$rrd_file:traffic_out:AVERAGE:step=1",
'CDEF:bitin=in,8,*',
'CDEF:bitout=out,8,*',
'CDEF:total=bitin,bitout,+',
'VDEF:pct_total=total,95,PERCENT',
'VDEF:pct_in=bitin,95,PERCENT',
'VDEF:pct_out=bitout,95,PERCENT',

To get In, Out and a 95th(In+Out).

But 95th(In+Out) although correct for what it is, it's not what I need as a total 95th.

I need it the conventional In values and Out values together "flattened" and 95th calculated off that. So that if 1 link is doing mostly Inbound with big In spikes this better 95th percentile will be a number closer to the average Inbound traffic, because most/all the top spikes will be inbound traffic outbound doesnt factor in much in the total usage. And similarly link 2 does mostly Outbound this 95th percentile would be a number closer to the Outbound average traffic.

Is there a way to do that with RRDtools alone at all, similar to the above command parameters?

In perl I've done something (dirty) like that but I don't align the time window (-a is not recognised by rrdtool fetch as a parameter) and for longer periods of time (say a month) I get few samples, so some reconciliation is hidden from me;

my ($start, $end, $rrd_file) = @_;

my @series = `rrdtool fetch -s $start -e $end $rrd_file MAX`;

my @all;
foreach my $line (@series) {
    if ($line !~ m/nan/ig && $line =~ m/(\d+): (\S+) (\S+)/) {
        my ($time, $in, $out) = ($1, $2, $3);

        push(@all, $in * 8, $out * 8);
    }
}

@all = sort {$b <=> $a} @all;

my $index_all = int(scalar(@all) * 0.05) + 1;

return sprintf("%.2f", $all[$index_all] / 1000 / 1000);

where

my $end = DateTime->now(time_zone => 'local')->truncate(to => 'minute');
my $start = $end->clone->subtract(hours => 168)->epoch();
$end = $end->epoch();

score 1 · Answer 1 · answered May 08 '16 at 20:00

1

I am not quite sure what you mean by 'flattened' but maybe you man to keep whichever is larger ?

"DEF:in=$rrd_file:traffic_in:AVERAGE:step=1",
"DEF:out=$rrd_file:traffic_out:AVERAGE:step=1",
'CDEF:bitin=in,8,*',
'CDEF:bitout=out,8,*'
'CDEF:bitmax=bitin,bitout,MAX',
'VDEF:pct_max=total,95,PERCENT',
'VDEF:pct_in=bitin,95,PERCENT',
'VDEF:pct_out=bitout,95,PERCENT'

answered May 08 '16 at 20:00

Tobi Oetiker

5,167
2
17
23

Yeah just not to care which is in or out but use all the datapoints from both in and out, line them up and get the 95th off that, will see how this looks but I suspect it will disregard the lower and I'll be endinb up with 95th(in) or 95th(out) depending on which was max – Recct May 09 '16 at 11:06
Gets me only the 9th of the bigger value, perhaps with some set operations I could combine the 2 data sources not sure how that might look like though – Recct May 09 '16 at 11:24
Essentially, calculate 95th Percentile on `in` and `out` series rather than the results of their averaging/ converting to bits – Recct May 09 '16 at 11:39
hihi there are many things you can calculate ... it relay depends on what you want :) First define the problem ... – Tobi Oetiker May 09 '16 at 11:47
Added a little programatic solution I did which I think some consider an accepted as "in+out" (although the `+` is misleading) standard of 95th percentile for bandwidth – Recct May 09 '16 at 16:16

Get 95th Percentile only through RRD

1 Answers1