0

We have a standard alerting structure in our organisation: Our app is hooked up with Prometheus and the alerts our received by Alertmanager and sent to our team in Slack. We have segregated our alerts as minor and major. I am trying to find a way so that I can quantify for a time range, which alert was fired by Alertmanager and how many times. Our corporate policy doesn’t allow me to write an app that can query Slack and just retrieve that information as that is the source of truth. So I tried to access this information via the Prometheus and Alertmanager api (v1 and v2) but am unable to do so. Prometheus queries show me all the alerts and this doesn’t match up to what we’ve received in Slack. And the Alertmanager query doesn’t allow me to filter on time (or a time range). Any help would be greatly appreciated.

Nikhil Sharma
  • 1
  • 1
    Try this link I found as a start? https://stackoverflow.com/questions/52480567/count-alerts-fired-by-prometheus – dtc Mar 10 '20 at 14:48
  • Thank you. That's definitely useful. However, is it possible to get this information straight from Alertmanager? That is the de-duped info and is exactly what has been fired. – Nikhil Sharma Mar 10 '20 at 23:23
  • if you have alerts, there's an "alerts" section in the UI they provide for you which points out which alerts are currently active as well as inactive. The alertmanager actually is not where you would go. It might be a bit confusing if you're new to it but prometheus alerts go from prometheus to the alertmanager and most of the times, you're almost never going to interact with the alertmanager. – dtc Mar 11 '20 at 03:14
  • Overall, there's prometheus and the alertmanager which are two separate processes. Since your app is already hooked up, it sounds like you can ask a coworker more about it OR search online for a way to setup your own prometheus and play around with that. – dtc Mar 11 '20 at 03:15
  • Thank you for your help – Nikhil Sharma Mar 11 '20 at 16:13

0 Answers0