0

Our ecosystem right now is graphite/grafana and we use the codahale metrics java library.

I define a counter

requestCounter = registry.counter(MetricNamespaces.REQUEST_COUNT);

and increment on every request hit to our app 

requestCounter.inc();

What we observerd with codahale is that, the counter s a cumalative value... When we look at the raw data in grafana, it is an increasing value over a period of time

What functions do I use in graphite so that I can get request count per min

I tried this 

alias(summarize(perSecond(sumSeries(app.request.count.*)), 
'1m', 'sum',     false), 'Request Count')

and also this 

hitcount(perSecond(app.request.count.*), '1m')

It doesn't seem right, Can someone please advice what is the recommended way and also if we can have codahale send just the raw data when incremented instead of a cumalative count

hackmabrain
  • 457
  • 1
  • 6
  • 21

1 Answers1

0

You should use nonNegativeDerivative function of the graphite API if you want to see the rate of a counter:

nonNegativeDerivative(sumSeries('app.request.count.*))

You need to notice that you also need to configure your graphite retention policy for your metrics. Otherwise, if the resolution of your metrics does not fit the way it's sent from codahale, you'll get weird unscaled results.

For example - in our company the codahale is configured to send data every two seconds. The graphite retention policy is 1 second for the first 6 hours and then 10 seconds. If we try to look at results beyond 6 hours, they're scaled incorrectly. I actually got to this question when trying to solve this issue. I'll update here when I have an answer.

Avi
  • 21,182
  • 26
  • 82
  • 121
  • for future readers: maybe you should add scaletosecond after your derivative (or nonnegativederivative) to normalize the data, or replace the derivation with perscond. – Reut Sharabani Oct 19 '21 at 11:13