1

I've built a bunch of munin plugins that monitor various back-end services. If the metrics dip below thresholds set in munin.conf, we're notified by email. However, if one of these services goes down completely, the plugin will fail and nobody gets notified!

I've followed the module-writing guide and added an exit code and message:

 sys.stderr.write('Error connecting to %s: %s\n' % (name, e))
 sys.exit(2)

But this only appears in the log. Nobody is watching the log.

Is there a way to make Munin alert on a complete plugin failure?

a paid nerd
  • 121
  • 5
  • 1
    I've not looked into munin plugins, but I would think could you just wrap the code that fails in a try..except and return a value below the threshold (a specific value for meaning service offline, e.g., -1) in the exceptional case. – JCallicoat Aug 23 '11 at 19:58
  • Heh, I thought of returning -1 as well. – a paid nerd Aug 23 '11 at 21:14
  • downside of that approach is it will screw up your averages. – Peter Green Aug 22 '16 at 12:50

1 Answers1

0

Are you open to alternatives? I believe that logcheck could solve your problem.

PythonLearner
  • 1,032
  • 2
  • 12
  • 31