using prometheus pod to monitor a golang webapp pod

Question

I have a golang webapp pod running in kubernetes cluster, and I tried to deploy a prometheus pod to monitor the golang webapp pod.

I specified prometheus.io/port: to 2112 in the service.yaml file, which is the port that the golang webapp is listening on, but when I go to the Prometheus dashboard, it says that the 2112 endpoint is down.

I'm following this guide, tried this thread's solution thread, but still getting result saying 2112 endpoint is down.

Below is the my service.yaml and deployment.yaml

apiVersion: v1
kind: Service
metadata:
  name: prometheus-service
  namespace: monitoring
  annotations:
      prometheus.io/scrape: 'true'
      prometheus.io/path: '/metrics'
      prometheus.io/port:   '2112'
spec:
  selector: 
    app: prometheus-server
  type: NodePort  
  ports:
    - port: 8080
      targetPort: 9090 
      nodePort: 30000
---
apiVersion: v1
kind: Service
metadata: 
  namespace: monitoring
  name: goapp
spec:
  type: NodePort
  selector:
    app: golang
  ports:
    - name: main
      protocol: TCP
      port: 80
      targetPort: 2112
      nodePort: 30001






apiVersion: apps/v1
kind: Deployment
metadata:
  name: prometheus-deployment
  namespace: monitoring
  labels:
    app: prometheus-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus-server
  template:
    metadata:
      labels:
        app: prometheus-server
    spec:
      containers:
        - name: prometheus
          image: prom/prometheus
          args:
            - "--config.file=/etc/prometheus/prometheus.yml"
            - "--storage.tsdb.path=/prometheus/"
          ports:
            - containerPort: 9090
          volumeMounts:
            - name: prometheus-config-volume
              mountPath: /etc/prometheus/
            - name: prometheus-storage-volume
              mountPath: /prometheus/
      volumes:
        - name: prometheus-config-volume
          configMap:
            defaultMode: 420
            name: prometheus-server-conf
  
        - name: prometheus-storage-volume
          emptyDir: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: monitoring
  name: golang
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: golang
    spec:
      containers:
        - name: gogo
          image: effy77/gogo2
          ports: 
            - containerPort: 2112
  selector:
    matchLabels:
      app: golang

I will try add prometheus.io/port: 2112 to the prometheus deployment part, as I suspect that might be the cause.

Did you make sure you goapp is exposing it's metrics as prometheus expects? What happens when you query the app at `:2112/metrics`? — Yaron Idan, Jan 30 '21 at 16:21
@YaronIdan when I query the golang app at /metrics, it displays the metrics. I guess I'm confused where to put the annotations, it looks like it needs to be put into the golang app section, I tried to put it under golang app's metadata, but the pod doesn't seem to appear in Prometheus dashboard. — allen, Jan 30 '21 at 16:27

score 2 · Answer 1 · answered Jan 30 '21 at 16:37

I was confused with where to put the annotations,got my clarifications from this thread, I needed to put it under the service's metadata that needs to be scraped by prothemeus. So in my case it needs to be in goapp's metadata.

apiVersion: v1
kind: Service
metadata: 
  namespace: monitoring
  name: goapp
  annotations:
      prometheus.io/scrape: 'true'
      prometheus.io/path: '/metrics'
      prometheus.io/port:   '2112'

using prometheus pod to monitor a golang webapp pod

1 Answers1