Stack Overflow
Stack Overflow

Questions tagged [prometheus-alertmanager]

The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts.

The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts.

Source code is available here: https://github.com/prometheus/alertmanager

753 questions
3
votes
1 answer

Creating alert using prometheus everytime there is an error

I am new to prometheus and alerting system. I have developed a microservice and added metrics code to get the total number of increments whenever there is an error. Now I am trying to create an alert so that whenever there is an increment in the…
Ashish Gupta
  • 83
  • 3
  • 7
3
votes
1 answer

Prometheus alert manager doesnt send alert k8s

Im using prometheus operator 0.3.4 and alert manager 0.20 and it doesnt work, i.e. I see that the alert is fired (on prometheus UI on the alerts tab) but I didnt get any alert to the email. by looking at the logs I see the following , any idea ?…
Rayn D
  • 579
  • 1
  • 13
  • 29
3
votes
1 answer

Configure Prometheus alerting rules combining the status of 2 different instances

I'm trying to configure into Prometheus Alerting Manager an alert that appears when the status of 2 different hosts is down. To better explain, I have these couples of hosts (host=instance): host1.1 host1.2 host2.1 host2.2 host3.1…
Sugaretto
  • 31
  • 2
3
votes
1 answer

How can we write alert rule comparing with the previous value for the prometheus alert rule

I am trying to setup an alert rule for elastic search number of nodes. If the number is less than the previous number , then i need an alert. I don't want to hard code the value. Since number of nodes are vary for each cluster. I am searching but…
Anil Kumar
  • 381
  • 1
  • 2
  • 9
3
votes
1 answer

alert when pod is restarted: Prometheus on windows containers

I have Prometheus setup for Windows containers which scrapes the metrics supported by the wmi_exporter. But I wanted some metrics that track the pod restarts, etc which are not part of that. I believe kube-state-metrics offers this functionality.…
sai guru datt manchikanti
  • 129
  • 1
  • 11
3
votes
1 answer

K8S - Not able to see alerts via - alertmanager

I've Prometheus operator which is working as expected https://github.com/coreos/prometheus-operator Now I want to apply the alert manager from scratch After reading the docs im came out with those yamls. but the problem is when I entered to the UI…
Jenny M
  • 923
  • 1
  • 14
  • 37
3
votes
1 answer

Prometheus kube_pod_container_status_waiting_reason not capturing pod CrashLoopBackOff reason

By definition, kube_pod_container_status_waiting_reason is supposed to capture reasons for a pod in Waiting status. I have several pods in my kubernetes cluster which are in CrashLoopBackOff but I dont see that reason captured by…
Preeti V
  • 93
  • 2
  • 9
3
votes
0 answers

Alertmanager prometheus, Sending to invalid slack channel

We have a multi tenant Kubernetes cluster and using Prometheus Alertmanager to send alerts to those tenants via slack. So we have config that includes this: slack_configs: - send_resolved: true channel: '{{ printf "topic-svc-%.11s" (index (index…
Steve Goodenough
  • 161
  • 1
  • 7
3
votes
1 answer

How can I send multiple alerts to one receiver group in Alertmanager?

I have looked at this question/answer here: Prometheus AlertManager - Send Alerts to different clients based on routes And it was a pretty good start for me, and I wish I could've commented a quick question to the Answerer there but I don't have the…
Cameron Rosier
  • 223
  • 3
  • 15
3
votes
1 answer

Prometheus alertmanager custom receiver

I'm currently trying to setup Prometheus and Alertmanager, the issue I have come to and haven't found solution yet, is that I want alert to be send to our custom application with oauth2 authorization. Is there any way how to build/configure own…
Kamil
  • 990
  • 1
  • 7
  • 24
3
votes
3 answers

How does Prometheus know when a pod crashed?

I understand with Prometheus we can set up alerting rules which can detect and raise an alert if a pod crashes. I want to understand how does Prometheus itself know when a pod crashed or is stuck in pending state. Does it know this when it is…
BlueChips23
  • 1,861
  • 5
  • 34
  • 53
3
votes
3 answers

Query on custom metrics exposed via prometheus node exporter textfile collector fails

I am new to prometheus/alertmanager. I have created a cron job which executes shell script every minute. This shell script generates "test.prom" file (with a gauge metric in it) in the same directory which is assigned to…
user_2011
  • 241
  • 1
  • 3
  • 10
2
votes
0 answers

How to send only 1 alert from prometheus for KubeJobFailed metric?

We monitor the KubeJobFailed via prometheus by the expression: kube_job_failed{job="kube-state-metrics",namespace=~".*"} > 0 As for now, we receiving multiple alerts for the same job (the job isn't get 'resolve' between the alerts). We would like to…
Moshiko Siyahu
  • 21
  • 2
2
votes
1 answer

Relabel configs in Prometheus alert manager

I have alert rules in Prometheus and I want to achieve that I don't see the cluster label in the alert but instead I see the env label. I tried to use this code but absolutely no result. - alert: SQL expr:…
Sina
  • 31
  • 2
2
votes
1 answer

Go Template if condition

how can i combine and and eq/ne functions together? I wrote this snippet {{ define "opsgenie.default.tmpl" }} {{.CommonLabels.alertname }} {{- range $i, $alert := .Alerts }} {{…
olti
  • 91
  • 1
  • 10
1 2 3
…
50 51