Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [prometheus]

The Prometheus monitoring system.

136 questions
2
votes
1 answer

Spikes on External Metric scales HPA when it shouldn't

I have a metric that I’m using for a HPA, the problem is that the metric has spikes and to avoid so, I’m using an average over time in a record rule on prometheus, but to export it to stackdriver(on gke) I’m using stackdriver-prometheus-sidecar and…
José Pedro Machado
  • 21
  • 3
2
votes
1 answer

Prometheus Alerting on NodeClockNotSynchronising for VMs

I'm trying to determine why this Alert (NodeClockNotSynchronising) is firing for a handful of VMs I've provisioned. (Not all just a few, which is strange) According to the metrics that are exported, I'm seeing: # HELP node_timex_sync_status Is clock…
Gerb
  • 123
  • 1
  • 4
2
votes
1 answer

Pod using Vernemq helm package cannot start

I'm using helm to install vernemq on my kubernetes cluster The problems is it can't start, I accepted the EULA Here is the log: 02:31:56.552 [error] CRASH REPORT Process <0.195.0> with 0 neighbours exited with reason:…
Lê Minh Quân
  • 301
  • 2
  • 13
2
votes
2 answers

What does it mean to have more than one instance of Prometheus in Kubernetes

Suppose I'm using a volume to persist my Prometheus data, I wonder if I can have more than one instance of it running to have high availability. I believe only one instance of Prometheus must be in charge of writing to the tsdb series, and having…
Ali Tou
  • 121
  • 1
  • 4
2
votes
1 answer

How to trigger alerts in Prometheus when specific users login to OpenShift or Kubernetes?

Using either kube_state_metrics or anything else I'd like to fire alerts in Prometheus AlertManager when a specific user logs in to the cluster, ie. kubeadmin or bob-smith. Or in other words: where in Kubernetes or OpenShift is the logged-in state…
funix
  • 21
  • 2
2
votes
1 answer

AWS CloudWatch metrics - are there convergence delays?

We are in the process of setting up aws cloudwatch exporter(https://github.com/prometheus/cloudwatch_exporter) to export cloudwatch metrics to our prometheus server. When reading through the documentation, I see a note about convergence of aws…
johnzac92
  • 21
  • 1
1
vote
0 answers

Convert json file to prometheus metrics

I have nginx json access log like this, a file that append sample like this to end of file. { "time": "2019-11-27T08:23:39+00:00", "msec": 1574843019.787, "ip": "91.133.248.69", "agent_name": "firefox", "agent_version": "70", "agent_os":…
Mohammad Hossein Fattahizadeh
  • 471
  • 1
  • 9
  • 27
1
vote
0 answers

Prometheus: how to stop alerting on a dead system

I've got my backup server reporting the age of the last successful backup to Prometheus, and I alert when that gets too old. I've just permanently shut down a server, and of course Prometheus is now alerting. What's the correct way to deal with…
Michael Kohne
  • 2,334
  • 1
  • 16
  • 29
1
vote
2 answers

Kubernetes: horizontal auto-scaling based on metrics in another namespace

I'd like to set up horizontal auto-scaling for a deployment based on the metrics of the ingress-controller deployed in another namespace. I have a deployment (petclinic) deployed in a certain namespace (petclinic). I have an ingress controller…
Volodymyr Melnyk
  • 577
  • 1
  • 5
  • 19
1
vote
2 answers

How to get CPU usage percentage for a namespace from Prometheus?

Our product lives in a Kubernetes cluster on our server. It is not in production yet, so there are multiple instances running in the cluster for different purposes, each in its own namespace. I need to run some load tests on one of the namespaces…
zslim
  • 121
  • 1
  • 1
  • 5
1
vote
1 answer

Query 5 minute average from Prometheus

I have a query to get used memory in Prometheus as a 0.0-1.0 percent. I can alert on this directly but don't want alerts on a short burst, only a high level over time or an average over time exceeding the limit. I was hoping to do this in the query,…
virullius
  • 1,048
  • 1
  • 9
  • 23
1
vote
0 answers

iptables rules for dockerized prometheus stack and swarm

OS: ubuntu Everything runns in containers (run via docker stack deploy) 2 servers running separate swarm server - running grafana + prometheus (ip 1.1.1.1) server - running node_exporter + cadvisor (ip 2.2.2.2) Need to setup IPTABLES: On 1st…
Palino1611
  • 11
  • 3
1
vote
1 answer

How to check to exist a unit systemd on the servers by Prometheus?

I have a service 'service_1' on the servers. And I need checking by Prometheus existing this service on the servers. node_systemd_unit_state showing only metrics for existing services. Maybe I can check existing file in…
tutunak
  • 63
  • 7
1
vote
1 answer

No prometheus in Gitlab Omnibus

Setup: Ubuntu 14.04 with all recent updates, Gitlab Omnibus version 11.7.3 (latest) ~$ sudo grep prometheus /etc/gitlab/gitlab.rb prometheus_monitoring['enable'] = true prometheus['enable'] = true prometheus['listen_address'] = ':9090' The…
wl2776
  • 181
  • 1
  • 9
1
vote
1 answer

kubernetes: measuring resource/limit efficiency

I'd like to get a feel for how 'efficient' my deployment/jobs are at consuming the resources they request. i.e., if a job that only ends up using 1CPU at peak ends up requesting 320, I'd like to have a dashboard/alert/metric to chase down rogue…
Eddie Parker
  • 571
  • 1
  • 4
  • 10
1 2
3
…
9 10