Stack Overflow
Stack Overflow

Questions tagged [prometheus-alertmanager]

The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts.

The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts.

Source code is available here: https://github.com/prometheus/alertmanager

753 questions
8
votes
3 answers

Alertmanager Match vs Match_re

match: [ : , ... ] # A set of regex-matchers an alert has to fulfill to match the node. match_re: [ : , ... ] My question is what is the difference between a match and a match_re statement? I have used…
zemmer W
  • 139
  • 1
  • 2
  • 7
7
votes
4 answers

Prometheus Absent function

I want to check if a certain metric is not available in prometheus for 5 minute. I am using absent(K_KA_GCPP) and giving a 5 minute threshold. But it seems I cannot group the absent function on certain labels like Site Id. Absent works if the metric…
Arnav Bose
  • 791
  • 4
  • 13
  • 27
7
votes
1 answer

How to get a pod's labels in Prometheus when pulling the metrics from Kube State Metrics

I have a Prometheus pod running along with my Kube-State-Metrics (KSM) pod. The KSM collects all the metrics from all the pods across all the namespaces in the cluster. Prometheus simply scrapes the metrics from KSM - this way Prometheus doesn't…
BlueChips23
  • 1,861
  • 5
  • 34
  • 53
6
votes
2 answers

How to overwrite alertmanager configuration in kube-prometheus-stack helm chart

I am deploying a monitoring stack from the kube-prometheus-stack helm chart and I am trying to configure alertmanager so that it has my custom configuration for alerting in a Slack channel. The configuration in the pod is loaded from…
everspader
  • 1,272
  • 14
  • 44
6
votes
1 answer

What does "prometheus-operated" service do?

Prometheus-operator seems to generate promethues-operated service which just points to Prometheus instance at port 9090. What does this service do? We define other services to point at our Prometheus cluster. What would be repercussions on removing…
Steve
  • 863
  • 3
  • 9
  • 21
6
votes
1 answer

How can I continuously poll JSON data in Prometheus?

I am new to prometheus so did some researched and found that it is helpful for monitoring application there are lot of examples of increasing counter to check api calls. My use case is somewhat different, I have a Rest API which is connecting to…
Shubham kapoor
  • 394
  • 3
  • 18
6
votes
2 answers

Alertmanager failing due to "unmarshal errors: line 19: field matchers not found in type config.plain"

I am trying to add custom alert-routing config to my alertmanager, deployed as a part of kube-prometheus-stack. But prometheus-operator pod, while trying to generate the alertmanager configmap, fails due to the following error: level=error…
Tapan Halani
  • 334
  • 1
  • 3
  • 11
6
votes
0 answers

Is it possible to have dynamic groupBy in Prometheus and Alertmanager?

In my Prometheus alert configuration, I have some great summary annotations, containing different labels. - alert: PodsPending expr: kube_pod_status_phase{phase="Pending"} > 0 labels: severity: high annotations: summary: "Namespace…
moffeltje
  • 4,521
  • 4
  • 33
  • 57
6
votes
1 answer

How do I use the hour() function of PromQL?

I am trying to set up an alert in the alert.rules file of Prometheus that fires only during specific periods of time. I've already tested the code block below without time restrictions inside the expr-tag and it works absolutely fine. As PromQL…
Lavair
  • 888
  • 10
  • 21
6
votes
2 answers

How to trigger alert in Kubernetes using Prometheus Alert Manager

I have setup kube-prometheus in my cluster(https://github.com/coreos/prometheus-operator/tree/master/contrib/kube-prometheus). It contains some default alerts like "CoreDNSdown etc". How to create my own alert? Could any one provide me sample…
JibinNajeeb
  • 784
  • 1
  • 10
  • 31
6
votes
4 answers

How to configure alerts in Prometheus for diskspace

We have prometheus running on Win Server box, and WMI exporter on a separate box(client). Able to read client metrics in Prometheus. Now the requirement is the moment Diskspace =>90 % , send an email alert, so that we can run a job to clean up…
Abraham Dhanyaraj Arumbaka
  • 859
  • 3
  • 11
  • 25
6
votes
2 answers

Environment variables in prometheus values

i would like to set environment specific values based on environment qa/prod in prometheus values file ## Additional alertmanager container environment variable ## For instance to add a http_proxy ## extraEnv: {}
shiv455
  • 7,384
  • 19
  • 54
  • 93
5
votes
2 answers

Alertmanager cluster status is "Disabled"

I am trying to configure alerts using Alert Manager with Prometheus (using Helm and Kubernetes). On Alert Manager UI, I am getting cluster status as disabled. How do I move it to a ready state? Attaching the image for the same
Prerna Singla
  • 51
  • 1
  • 2
5
votes
1 answer

Where to get which rules that are failed in Prometheus?

I got this alert: Alert: PrometheusRuleFailures - critical Description: Prometheus monitoring/prometheus-prometheus-kube-prometheus-prometheus-0 has failed to evaluate 30 rules in the last 5m. Details: • alertname: PrometheusRuleFailures •…
Kokizzu
  • 24,974
  • 37
  • 137
  • 233
5
votes
1 answer

Missing labels in prometheus alerts

I'm having issues with Prometheus alerting rules. I have various cAdvisor specific alerts set up, for example: - alert: ContainerCpuUsage expr: (sum(rate(container_cpu_usage_seconds_total[3m])) BY (instance, name) * 100) > 80 for: 2m labels: …
dywan666
  • 385
  • 8
  • 14
1
2
3
…
50 51