1

I'm writing a daemon that will continuously run in background and do it's thing. I would like to monitor the performance and various other variables. This will include things like web request latency, errors, number of processed items and so on.

The data should then be available for review, along with graphs, reports (averages, outliers) and alerts when limits are exceeded.

Performance counters on Windows are the closest thing to this I could find. You can define your own counters and then easily access them from the code.

Note that I'm not looking for "performance" counters as those are named, it's more like application counters - i.e. I don't care about number of executed instructions and cache misses, but about custom counters. A StackOverflow questions similar to this where the responses missed the point is here: Application counters in Linux? (and OSX?)

I've looked at Cacti (and alternatives), but they seem to run a script on a schedule which provides the data. That means I would have store the counters in my daemon, along with a thread waiting for some kind of IPC call. Then I would need to create a second application/script which would connect via this IPC interface and retrieve the stored values.

Is there a better way? Some C library that will make it possible to increment a counter with a single function call and it'll automatically get processed via the frontend?

Community
  • 1
  • 1
lacop
  • 2,024
  • 4
  • 22
  • 36

2 Answers2

3

statsD is a good place to start to collect metrics.

statsD will let you send counters directly from your application without incurring any significant overhead. Your code generates counters as it executes and you need not keep track of it in your code.

If you want a package that has all-in-one (but no persistence), you can use pup

If you need persistence and graphing you may want to look at graphite.

graphite will connect to statsd and let your graph any of the counters you generate. It's fairly lightweight and the combination is arguable easier to deploy and maintain than something like Nagios (which is much closer to cacti in terms of deployment and learning curve).

If you want something more integrated with persistence and an API, there a number of services to do the graphing for you. I've limited myself to open-source software here.

Alexis Lê-Quôc
  • 1,023
  • 7
  • 8
  • Thank you, this looks like it should work. C API for statsD seems pretty rough but it's just a simple UDP messaging system so I could write my own in the worst case. I'll leave this question open for a while to see if there are any other options. – lacop Aug 17 '13 at 15:46
0

I think You are reinventing the wheel.

try Nagios http://nagios.org/

-You can write any custom check for nagios and monitor performance of anything You need - like Your apps. -You can customize Your alerts and when and how they are sent do who/where. -You can create relationships between services/hosts etc...

Plugins/extentions: http://exchange.nagios.org/

Bartłomiej Zarzecki
  • 141
  • 3
  • I'm trying to avoid reinventing. Read the question, the form of reporting Cacti (and Nagios) uses require me to write custom RPC service inside the daemon. – lacop Aug 17 '13 at 15:39