Stack Overflow
Stack Overflow

Questions tagged [prometheus-alertmanager]

The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts.

The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts.

Source code is available here: https://github.com/prometheus/alertmanager

753 questions
2
votes
1 answer

Dynamically Adding 'team' Label to Alerts in Prometheus Using Regex

I'm working with Prometheus alerts, and I would like to dynamically add a 'team' label to all of my alerts based on a regex pattern. I have an example alert: expr:…
TomerA
  • 23
  • 5
2
votes
1 answer

Prometheus alert expression during the special period of time

I got the following Prometheus alert: - alert: HostHighCpuLoadTSDB expr: (100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle",common_name="stg-tsdb-eks"}[2m])) * 100) > 5) and (hour() < 8 or hour() > 9) for: 5m …
Oleksii K
  • 23
  • 3
2
votes
0 answers

JIRA automation to resolve tickets

I was looking a way to create a JIRA automation that resolves an issue when an issue with the prefix "[RESOLVED] " created. A bit of background, the issues are created through email from AlertManager. More specific Alertmanager sends a "[FIRING:1]…
Thanos Mourtzoukos
  • 175
  • 1
  • 8
2
votes
1 answer

Mock time in prometheus alert unit test

Morning everyone, I have a prometheus alert that looks like that : whatever_expr > 0.10 AND ON() absent(hour() >= 2 <= 3) That excludes alert during some maintenance schedule. Now that I want to unit test it, I can't find any information to mock…
Boris Le Méec
  • 2,383
  • 1
  • 16
  • 31
2
votes
0 answers

Using repetitive blocks in global or as variable in alertmanager.yml config file

I have to send different alerts in 20 Slack channels based on some filter. My Alertmanager config file looks like (only two receiver as example): global: resolve_timeout: 1m slack_api_url: '' templates: -…
bobs
  • 55
  • 8
2
votes
1 answer

Loki groups rules get content log

I want to notify when a log type error occurs and send its log content, will it be possible to add it in the labels or annotations of the loki rules and send it to alertmanager? This would be my test rule, but I still don’t see how to add the…
MiguEl
  • 21
  • 2
2
votes
0 answers

Kubernetes CronJob - Prometheus alerts

I am trying to figure out how to create Prometheus alerts for my kubernetes cronjob for the following scenarios using kube-state-metrics. If my cronjob fails, send an alert, after a minute if it's still failing or another failed cronjob exists,…
Matt
  • 51
  • 3
2
votes
0 answers

Unable to generate slack alert

I am trying to setup slack alerting for when a probe for an endpoint is down. I am using black box exporter for this. First ive managed to get the alerts for when a probe returns failure by using the following config prometheusSpec: …
Jason Stanley
  • 169
  • 1
  • 10
2
votes
2 answers

Testing Grafana alert returns "Failed to test the rule" with 500 - Internal Server Error response

I'm trying to set up alerts on one of my graphs. I'm using AMG (Amazon Managed Grafana). However, I'm getting "Failed to test the rule" notification. When I inspect HTTP response, it shows Status Code: 500 Internal Server Error {"message":"Failed…
TariqN
  • 668
  • 2
  • 9
  • 27
2
votes
2 answers

Not able to install prometheus on Mac os

I have downloaded prometheus-2.36.2.darwin-amd64.tar.gz file in my Mac system with a M1 processor. But I am unable to install and run prometheus. Please let me know the steps to install the same.
Eli Johnes
  • 301
  • 3
  • 13
2
votes
3 answers

Prometheus not sending alarms to Alertmanager

although I can see my alarms on prometheus, I cannot view alarms via alertmanager, although their status is Firing. My settings; prometheus.yml config # Alertmanager configuration alerting: alertmanagers: - static_configs: - targets: …
tcblue
  • 179
  • 3
  • 12
2
votes
0 answers

Alertmanager docker. Set telegram_configs bot_token and chat_id from environment

I have alertmanager/config.yml receivers: - name: 'telegram' telegram_configs: - send_resolved: true api_url: 'https://api.telegram.org' bot_token: '123456:ASDFGH-:LKHHGFGD-WERWERWER' …
KoVaLsKy
  • 121
  • 3
2
votes
1 answer

Alertmanager configuration does not get updated when values.yml have changed

I'm trying to configure Alertmanager with Mattermost. For the whole monitoring and alerting system we're using the Helm rancher-monitoring charts. When using the default values.yml file from this version of the chart everything will be deployed…
Lucas Scheepers
  • 505
  • 6
  • 24
2
votes
1 answer

Dynamically process template map values

I have annotations object like this: Annotations: map[string]string{ "common": "Some GPM consistency checkers have failed.", "t_0": "Title1", "t_1": "Title2", "v_0": "V1", "v_1": "V2", I want to print it in…
user122222
  • 2,179
  • 4
  • 35
  • 78
2
votes
0 answers

How independently import files in values.yaml

I'm using the prometheus stack project to implement via helm with kubernetes, but I would like to customize some rules. I'm trying to use the additionalPrometheusRules parameter for this, but I'm facing syntax errors inside it, here's my…
Luis Henrique
  • 701
  • 15
  • 36
1 2 3
…
50 51