Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [prometheus]

The Prometheus monitoring system.

136 questions
0
votes
0 answers

Prometheus scraping halted for last 20 minutes

My prometheus dashboard has values missing for the last 20 minutes, no matter when I query (container_cpu_usage_seconds_total) it. What could be the reason behind this? Screenshot of dashboard
theodre7
  • 1
0
votes
0 answers

What is the best way to set up Prometheus + Alertmanager with a HA

I have a monitoring solution that uses Prometheus as a scraper and a data storage, Grafana as a visualiser and Alertmanager as an alerting tool. This all is running on a single server. However, there's an issue with this approach. If a server that…
serge1peshcoff
  • 413
  • 1
  • 4
  • 8
0
votes
1 answer

promtool tsdb create-blocks-from openmetrics doesn't show any data in Prometheus

I'm trying to backfill Prometheus with old data. My input data is: $ cat /etc/prometheus/backfill.txt # HELP mymetric My metric. # TYPE mymetric counter mymetric{code="200",service="user"} 123 1665824331000 mymetric{code="500",service="user"} 456…
Bruno Pérel
  • 131
  • 1
  • 1
  • 5
0
votes
1 answer

Prometheus: Add insecure_skip_verify via annotation or scrape config adaption for kubernetes pods

I`ve running a kubernetes cluster with a deployment of some pods. One pod provides metrics on a https secured endpoint. The problem is, that this pod create and use his own self-signed certificate and prometheus does not trust them. It's ok for me,…
Volker Raschek
  • 347
  • 1
  • 5
  • 17
0
votes
0 answers

Prometheus query to identify active series if a datapoint is sent in the last 15-30mins

I am trying to identify my Prometheus active series if a datapoint is sent in the last 15-30mins. As I understand it, this query prometheus_tsdb_head_series gives me active series if a datapoint is sent in the last 1-3 hours. Can anyone help with…
qu_001
  • 1
0
votes
0 answers

GKE metrics about CPU usage of `kube-apiserver`

I'm trying to benchmark the load on the Kubernetes API that two different deployments cause. I have tried the two following ways of accomplishing this: Evaluate the Prometheus metric apiserver_request_total. Unfortunately, this does not take into…
Luca Mertens
  • 1
  • 2
0
votes
0 answers

servicemonitor doesnot add to prometheus-operator targets

I have deployed a Prometheus-operator on the k8s cluster. Everything works well, but I want to monitor MySQL pods in another namespace. I create mysqld-exporter pod and svc for it in MariaDB namespace and a servicemonitor for it in the monitoring…
mona moghadampanah
  • 9
  • 1
  • 3
0
votes
1 answer

prometheus field remote_write not found in type config.plain

I need help using prometheus remote_write and remote_read to/from influxdb. I got error below: # promtool check config /etc/prometheus/prometheus.yml Checking /etc/prometheus/prometheus.yml FAILED: parsing YAML file /etc/prometheus/prometheus.yml:…
sajr
  • 1
  • 1
  • 1
0
votes
0 answers

How to get Kafka consumer metrics with JMX exporter for Prometheus?

Kafka version: 3.1 I didn't find a way to get Kafka consumer metrics as described in Kafka documentation from my Kafka servers. Even if I leave the "rules" section empty in the config of the JMX exporter there are no "kafka_consumer_*" metrics…
André Veelken
  • 5
  • 2
0
votes
1 answer

Disabling email for Prometheus Alertmanager based on severity

We are monitoring a HPC computing cluster using a combination of Prometheus, Alertmanager and Grafana. On our machines, things like SWAP memory filling up to essentially the limit happen frequently, and while it is useful to see the corresponding…
ml4294
  • 101
  • 2
0
votes
1 answer

Change retention time on prometheus

So I am running Prometheus on an Ubuntu server and started running it a while ago by just running sudo ./prometheus However, I wanted to append --storage.tsdb.retention.time=1y to the end of this command. How can I change the storage retention…
AsapHogFtw
  • 1
0
votes
1 answer

webhook MS Teams integration with Prometheus - request failed

I'm struggling with Microsoft Teams/Prometheus integration on K8s cluster. I used helm to start all components. I have correctly working Prometheus and Alertmanager. It seems that all works fine. Prometheus communicate with Alertmanager. Then…
RedBluff
  • 31
  • 1
  • 3
0
votes
1 answer

Nginx reverse proxy when port is already in use?

i am trying to close port with basic authentication (for pushgateway of prometheus), so not a big specialist in nginx, so could someone please give me and advice where i am is wrong? I have 9091 port, that should be closed from outside in front of…
Nigrimmist
  • 103
  • 3
0
votes
1 answer

Debug Prometheus Blackbox Exporter http_2xx probs

We are using the Prometheus Blackbox Exporter (blackbox_exporter, version 0.19.0) to check HTTP endpoints. Now we need to implement a check with the fail_if_body_matches_regexp. The check should fail when a certain word exists inside the html…
Patrick
  • 121
  • 8
0
votes
0 answers

microk8s: pod resource usage metrics not available from all nodes

I am running microk8s v1.22/stable on a Linux cluster with 11 nodes. I have enabled the metrics-server plugin and installed Prometheus via the Helm chart with nodeExporter and kubeStateMetrics enabled. I get all sort of other metrics, but I have…
mhusaini
  • 101
  • 1
1 2 3
…
9 10