Prometheus configuration to scrape kube pod metrics

Question

I have the below prometheus configuration to fetch the kubernetes pods metrics but however , in prometheus i could see only 2 metrics related to pods which Kube_pod_CPU, Kube_pod_memory but not any other metrics related to pods. Any help would be appreciated if there is any configuration issue.

- job_name: 'kubernetes-pods'
  scrape_interval: 15s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  kubernetes_sd_configs:
   - role: pod
     namespaces:
       names: ["namespace-name"]
  relabel_configs:
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
    action: keep
    separator: ":"
    replacement: $1
    regex: true
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
    action: replace
    separator: ":"
    replacement: $1
    target_label: __metrics_path__
    regex: (.+)
  - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
    action: replace
    separator: ":"
/prometheus-kubernetes.yml
    regex: (.+):(?:\d+);(\d+)
    #regex: ([^:]+)(?::\d+)?;(\d+)
    replacement: ${1}:${2}
    target_label: __address__
  - source_labels: [__meta_kubernetes_pod_container_name]
    action: keep
    regex: prometheus.*
  - action: labelmap
    separator: ":"
    replacement: $1
    regex: __meta_kubernetes_pod_label_(.+)
  - source_labels: [__meta_kubernetes_namespace]
    separator: ":"
    replacement: $1
    action: replace
    target_label: kubernetes_namespace
  - source_labels: [__meta_kubernetes_pod_name]
    separator: ":"
    replacement: $1
    action: replace
    target_label: kubernetes_pod_name

Thanks

score 0 · Answer 1 · answered May 26 '22 at 19:39

> You have to add this three annotation to your deployment:

prometheus.io/scrape: 'true'
prometheus.io/path: 'metrics'
prometheus.io/port: '80'
How it will work?

Look at the kubernetes-pods job of config-map.yaml you are using to configure prometheus,



- job_name: 'kubernetes-pods'

        kubernetes_sd_configs:
        - role: pod

        relabel_configs:
        - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
          action: keep
          regex: true
        - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
          action: replace
          target_label: __metrics_path__
          regex: (.+)
        - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
          action: replace
          regex: ([^:]+)(?::\d+)?;(\d+)
          replacement: $1:$2
          target_label: __address__
        - action: labelmap
          regex: __meta_kubernetes_pod_label_(.+)
        - source_labels: [__meta_kubernetes_namespace]
          action: replace
          target_label: kubernetes_namespace
        - source_labels: [__meta_kubernetes_pod_name]
          action: replace
          target_label: kubernetes_pod_name

Prometheus configuration to scrape kube pod metrics

1 Answers1