0

I am getting metrics exposed by kube-state-metrics by querying Prometheus-server but the issue is I am getting duplicate metrics with difference only in the job field. . I am doing query such as :

curl 'http://10.101.202.25:80/api/v1/query?query=kube_pod_status_phase'| jq

The only difference is coming the job field. Metrics coming when querying Prometheus-Server

All pods running in the cluster: https://i.stack.imgur.com/WxNXz.jpg

Any help is appreciated.

Thank You

prometheus.yml

global:
  scrape_interval:     15s
  evaluation_interval: 15s

rule_files:
  # - "first.rules"
  # - "second.rules"

scrape_configs:
  - job_name: prometheus
    static_configs:
      - targets: ['localhost:9090']
S4H
  • 28
  • 4
  • Could you share your Prometheus configuration? (prometheus.yaml) – SYN Jun 22 '21 at 19:31
  • @SYN I am making use of value.yml file that is present here : https://github.com/prometheus-community/helm-charts/blob/main/charts/prometheus/values.yaml. – S4H Jun 22 '21 at 19:46
  • @SYN I have added the prometheus.yml file in the post. – S4H Jun 22 '21 at 20:04

2 Answers2

1

You are running (or at least ingesting) two copies of kube-state-metrics. Probably one you installed and configured yourself and another from something like kube-prometheus-stack?

coderanger
  • 52,400
  • 4
  • 52
  • 75
  • I had only 1 instance of the prometheus-kube-state-metrics which comes by default with prometheus when setting it up through the helm charts. But I am now getting the appropriate 1 type of metrics only. What I did was removing the scrap config of prometheus-kube-state-metrics from the value.yml and defining that in the config file i.e. prometheus.yml. For now it's working fine. – S4H Jun 22 '21 at 21:06
  • It doesn't come with Prometheus, you installed it via Helm from the looks of thing. The only thing that comes with Prometheus is Prometheus. – coderanger Jun 22 '21 at 22:57
0

I was able to get what I wanted eventually. What I did was to remove the scraping config of prometheus-kube-state-metrics from the value.yml and defining that in the config file i.e. prometheus.yml. For now it's working fine. Thank You @SYN and @coderanger for the help.

DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
S4H
  • 28
  • 4