Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [prometheus]

The Prometheus monitoring system.

136 questions
1
vote
1 answer

Prometheus alert not fired

I have setup 5 alerts in my Prometheus setup. 3 of them work as expected. However, I have 2 that are never triggered. I am really confused and I need some help here. So, the 2 rules that do not work are: alert: CriticalDiskSpace expr:…
Peter
  • 822
  • 2
  • 10
  • 23
1
vote
1 answer

Prometheus with remote_write and backups

I'm building a prometheus stack that uses the remote_read/remote_write feature for long term storage, and I have some questions about if/how prometheus' tsdb needs to be backed up, given that it is using long term storage. 1) Do I need to back up…
user307927
  • 133
  • 1
  • 4
1
vote
0 answers

Is there possibility in Prometheus alerts calculate another alert status?

I want to make an ALERT with IF that contains expression that depends on another alert status. Is that ever possible somehow? I know about alert inhibition with alermanager but I need not to firing some alerts at all in some cases. Prometheus…
Shtlzut
  • 290
  • 3
  • 9
1
vote
0 answers

my prometheus dashboard shows 0 on node_export instance

I have installed prometheus on a server and node_exporter on another server I configured my prometheus.yml to interact with the node exporter The connection works fine but the up query shows 0 which means that the instance is not healthy. The both…
Samson
  • 113
  • 5
1
vote
2 answers

Monitor Azure Kubernetes Service (AKS) with Prometheus servers running independently

Can I deploy Prometheus to each Azure Kubernetes Service (AKS), and monitor each Prometheus from a Prometheus created by a virtual machine outside the AKS? In other words, I want to connect the Prometheus in the center with all Prometheus in each…
gotothesky
  • 13
  • 3
1
vote
1 answer

Grafana graph value as a percentage of the initial value

I am new to grafana & prometheus. I want to graph the change in a value (here disk space used), but as a percentage of the initial value, which one commonly sees for inflation charts. For example, this chart of inflation has every line starting at…
Amandasaurus
  • 31,471
  • 65
  • 192
  • 253
1
vote
1 answer

Ceph alertmanager config

I've installed ceph using cephadm Including monitoring stack prometheus, alertmanager, node-exporter Currently I'm trying to add a telegram receiver (Telegram is supported from v0.24.0, so I've manually updated mgr/container_image_alertmanager fro…
SergkeiM
  • 63
  • 7
1
vote
2 answers

Grafana pod is not running, how to fix that?

I have deployed grafana in eks using the steps provided in this link After deployment of grafana, the pod is not in running state. kubectl get po -n grafana NAME READY STATUS RESTARTS AGE grafana-6db7758575-pfqdg 0/1 …
user2331760
  • 155
  • 4
  • 12
1
vote
0 answers

Hostname instead of IPs for panel legends in Grafana with Prometheus metrics

I want to monitor different servers with Prometheus fed by exporters like node-exporter or postgres-exporter. The metrices should be presented in Grafana as dashboards and alerts. The exporters will be imported by following targets in the…
Vaiaro
  • 113
  • 1
  • 6
1
vote
1 answer

Alertmanager telegram config chat_id and cannot unmarshal errror

I am trying to configure alertmanager to send alerts to my telegram group. Following the configuration I have: global: resolve_timeout: 5m route: group_by: - job group_interval: 5m group_wait: 30s receiver: "telegram" repeat_interval:…
Jose
  • 21
  • 1
  • 2
1
vote
1 answer

Still can't connect from my grafana docker container to prometheus

...after having followed a bunch of tutorials, e.g. https://medium.com/@TimvanBaarsen/how-to-connect-to-the-docker-host-from-inside-a-docker-container-112b4c71bc66 I started the grafana container with: %box docker run -d -p 3000:3000…
transient_loop
  • 499
  • 1
  • 4
  • 15
1
vote
0 answers

Prometheus alert rule test fails with long intervals

I've got an alert rule: groups: - name: somename rules: - alert: CertificateExpiry expr: certificate_expires_in_days < 20 for: 1h labels: …
badbishop
  • 928
  • 4
  • 12
  • 21
1
vote
1 answer

Can I tail the log of ONE container in a docker-compose group?

I'm still struggling to understand why my ipmi_exporter container will not run in the same "pod" as my Prometheus, Grafana, snmp_exporter and alertmanager. When I bring up the "pod," docker-compose says ipmi_exporter has started but docker-compose…
mr.zog
  • 923
  • 3
  • 20
  • 39
1
vote
1 answer

Why does my prometheus dashboard only show 2 weeks of data when I have longer term data available?

I have Prometheus configured with Thanos. I'm using the Node Exporter Full dashboard to view data from the node exporter. Regardless of what time range I select (e.g., now-20d), the dashboard only shows 2 weeks of data: If I go to the "Explore" UI…
larsks
  • 43,623
  • 14
  • 121
  • 180
1
vote
1 answer

Why K8S statefulsets volumeClaimTemplates status is pending , but the pod, pvc, pv are all fine?

I use nfs-subdir-external-provisioner as automatic pv provisioner for my promethues (by promethues-operator ) I have created our sts,pod,pvc,pv successfully and everything looks fine. But if I use kubectl get sts -o yaml , the status is still …
Jeffery
  • 13
  • 3
1 2 3
…
9 10