Expose grafana publicly using istio

Question

We are using Prometheus operator and we need to expose Grafana publicly (outside) using istio, https://github.com/helm/charts/tree/master/stable/prometheus-operator

Normally when I have application which I need to expose publicly with istio, I adding something like following to my micro service and it works and exposed outside.

service.yaml

apiVersion: v1
kind: Service
metadata:
  name: po-svc
  namespace: po
spec:
  ports:
    - name: http
      port: 3000
      targetPort: 3000
  selector:
    app: myapp  //I take the name from deployment.yaml --in the chart NOT SURE WHICH VALUE I SHOULD TAKE FROM THE CHART---

And add a virtual service

virtualservice.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: po-virtualservice
  namespace: po
spec:
  gateways:
    - gw-system.svc.cluster.local
  hosts:
    - po.eu.trial.appos.cloud.mvn
  http:
    - route:
        - destination:
            host: po-svc
            port:
              number: 3000

Then I was able to access to my application publicly.

Now I want to the same for Grafana from the prometheus operator chart

in the values.yaml there is service entry

https://github.com/helm/charts/blob/master/stable/prometheus-operator/values.yaml#L576 However not sure if it should replace the service.yaml and if yes how to fill the data like app: myapp (which in regualr application I take from the deployment.yaml the `name' field) to be the grafana that the service have the reference to Grafana application

in addition, in the virutalservice.yaml there is a reference to the service (host: po-svc)

My question is: How should I fill those two values and be able to expose Grafana using istio ?

Btw, if I change the values from the chart to LoadBalancer like below, im getting a public url to access outside, however I want to expose it via istio.

  service:
    portName: service
    type: LoadBalancer

update

I've created the following virtual service

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: po-virtualservice
  namespace: po
spec:
  gateways:
    - gw-system.svc.cluster.local
  hosts:
    - po.eu.trial.appos.cloud.mvn
  http:
    - route:
        - destination:
            host: po-grafana. // This is the name of the service that promethues operator created when applying the chart .
            port:
              number: 3000

and update the values.yaml like following

  service:
    portName: service
    port: 3000
    targetPort: 3000

Now when I hit the browser for the application url (po.eu.trial.appos.cloud.mvn) I got error

upstream connect error or disconnect/reset before headers. reset reason: connection termination any idea what could be the problem? how should I trace this issue ?

I would think(not sure 100%) I may be missing something on the service config in the chart but not sure what...

I've found this post which have similar error: (but not sure we have the same issue)

https://github.com/istio/istio/issues/19966

However not sure how should I add the nameport to the chart yaml service definition

Answer 1

There is a working example for istio with version 1.7.0

istioctl version
client version: 1.7.0
control plane version: 1.7.0
data plane version: 1.7.0 (1 proxies)

1.I have used helm fetch to get prometheus operator.

helm fetch stable/prometheus-operator --untar

2.I changed these in values.yaml.

Grafana Service.

service:
  portName: http-service
  port: 3000
  targetPort: 3000

Grafana host.

hosts:
  - grafana.domain.com

3.I have created po namespace and installed prometheus operator

kubectl create namespace po
helm install prometheus-operator ./prometheus-operator -n po

4.I have checked the grafana service name with

kubectl get svc -n po
prometheus-operator-grafana                    ClusterIP

5.I have used below yamls for istio, used grafana service name which is prometheus-operator-grafana as my virtual service and destination rule host.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: grafana-gateway
  namespace: po
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http-grafana
      protocol: HTTP
    hosts:
    - "grafana.domain.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: grafana-vs
  namespace: po
spec:
  hosts:
  - "grafana.domain.com"
  gateways:
  - grafana-gateway
  http:
  - route:
    - destination:
        host: prometheus-operator-grafana.po.svc.cluster.local
        port:
          number: 3000
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: grafana
  namespace: po
spec:
  host: prometheus-operator-grafana.po.svc.cluster.local
  trafficPolicy:
    tls:
      mode: DISABLE

5.Test with curl, it's 302 instead of 200 as we have to login.

curl -v -H "host: grafana.domain.com" xx.xx.xxx.xxx/

GET / HTTP/1.1
> Host: grafana.domain.com
> User-Agent: curl/7.64.0
> Accept: */*
>
< HTTP/1.1 302 Found

---

Let me know if it worked or if you have any other questions. Maybe there is a problem with the 1.4.3 version you use.