1

I have a lot of data collected in rrd files. Does anyone know of a script, or part of the rrd toolset that will show 'spikes'.

Basically I would like to give it a window specified by start time and end time. Then maybe a threshold above the average for that would be considered a spike. Then I would like the tool to print out the time stamps for those spikes and the values.

I know I could make some graphs and look at them, but that's not really what I am after with this. If nobody has a solution, I will probably whip something up in Perl, so any suggestions are also welcome.

Kyle Brandt
  • 83,619
  • 74
  • 305
  • 448

2 Answers2

2

I've used the later version of removespikes from RRDtool's contrib directory in the past to find and remove spikes over a certain percentage or value.

I'm not sure whether it will print the values as you need them out of the box. But if not, the perl should be the exact basis of what you need.

Dan Carley
  • 25,617
  • 5
  • 53
  • 70
  • at first it seems this directs the opposite of what Kyle is asking about, but after re-reading, perhaps you are pointing him to the removespikes script to determine how it detects them in the first place? The general problem with spike removal is that they are usually designed to remove spikes caused by counter roll-over, rather than actual spikes in the data (a counter-roll-over spike is usually looks like a multi-million unit spike as compared to a spike simply 10x normal). – ericslaw Jul 27 '09 at 17:14
  • `removespikes` is intended to be run to remove values above value or percentage above the baseline value - not just for roll-overs. My suggestion is to either use it in "dry run" mode, or modify it, to print the values instead of removing them. – Dan Carley Jul 27 '09 at 17:23
1

looking for spikes is an interesting problem in general. is your data noisy?

Try using the TREND modifier like this

CDEF:trendvar=oldvar,3600,TREND

to get a rolling 1hour average (or whatever looks like a good smoothing interval).

use RPN to compare that to your current value and use another CDEF to highlight the spike.

I have found limited value in percentage based banding around a moving metric.

I've built interactive cgi's to help explore up/down spike detection using RRDtool and jQuery.

ericslaw
  • 1,572
  • 2
  • 13
  • 15
  • Not too noisey. I use this with Nagios. If someone tells me, we have a problem around x time. I could use this to check lots of different services that maybe have had an issue, but not enough of an issue to trigger an alert. Thanks for the tip, will probably use that for the script, if I end up making it:-) – Kyle Brandt Jul 27 '09 at 17:32
  • Once you have graphs that clearly indicate spikes, then you can use the XPRT (export) directive (instead of LINEe or AREA) to dump in CSV format... Of course you could also just call 'fetch' and do thr trend calc yourself (think exponential moving average). – ericslaw Jul 28 '09 at 03:05