Customizing Istio metrics results in duplicated metrics

Question

I want to add request.url_path and request.method information to the istio_request_total metric by using an EnvoyFilter.

Istio Version: 1.8.5

Using my EnvoyFilter duplicates istio_requests_total metric by adding envoy_request_path__$(path-seperated-with-dashes)__ prefix.

Below is the EnvoyFilter I use

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: customize-requests-total-metric
spec:
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_OUTBOUND
        listener:
          filterChain:
            filter:
              name: envoy.filters.network.http_connection_manager
              subFilter:
                name: istio.stats
        proxy:
          proxyVersion: ^1\.8.*
      patch:
        operation: REPLACE
        value:
          name: istio.stats
          typed_config:
            "@type": type.googleapis.com/udpa.type.v1.TypedStruct
            type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
            value:
              config:
                configuration:
                  "@type": type.googleapis.com/google.protobuf.StringValue
                  value: |
                    {
                      "debug": "true",
                      "stat_prefix": "istio",
                      "metrics": [
                        {
                          "name": "requests_total",
                          "dimensions": {
                            "request_path": "request.url_path",
                            "request_method": "request.method"
                          }
                        }
                      ]
                    }
                root_id: stats_outbound
                vm_config:
                  code:
                    local:
                      inline_string: envoy.wasm.stats
                  runtime: envoy.wasm.runtime.null
                  vm_id: stats_outbound
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        listener:
          filterChain:
            filter:
              name: envoy.filters.network.http_connection_manager
              subFilter:
                name: istio.stats
        proxy:
          proxyVersion: ^1\.8.*
      patch:
        operation: REPLACE
        value:
          name: istio.stats
          typed_config:
            "@type": type.googleapis.com/udpa.type.v1.TypedStruct
            type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
            value:
              config:
                configuration:
                  "@type": type.googleapis.com/google.protobuf.StringValue
                  value: |
                    {
                      "debug": "true",
                      "stat_prefix": "istio",
                      "metrics": [
                        {
                          "name": "requests_total",
                          "dimensions": {
                            "request_path": "request.url_path",
                            "request_method": "request.method"
                          }
                        }
                      ]
                    }
                root_id: stats_inbound
                vm_config:
                  code:
                    local:
                      inline_string: envoy.wasm.stats
                  runtime: envoy.wasm.runtime.null
                  vm_id: stats_inbound

This duplicates istio_requests_total metric by adding envoy_request_path__$(path-seperated-with-dashes)__ prefix. Below is an example of it.

# TYPE envoy_request_path____catalog_lint___istio_requests_total counter
envoy_request_path____catalog_lint___istio_requests_total{response_code="200",reporter="destination",source_workload="my-service",source_workload_namespace="my-namespace",source_principal="unknown",source_app="my-service",source_version="unknown",destination_workload="api-linter",destination_workload_namespace="my-namespace",destination_principal="unknown",destination_app="my-service",destination_version="unknown",destination_service="my-service",destination_service_name="my-service",destination_service_namespace="my-namespace",request_protocol="http",response_flags="-",grpc_response_status="",connection_security_policy="none",source_canonical_service="my-service",destination_canonical_service="api-linter",source_canonical_revision="latest",destination_canonical_revision="latest",request_method="GET"} 4204

** My question is, how can I add request path as a label to the main istio_requests_total metric like I did for request_method="GET" in the example above?**

score 0 · Accepted Answer · answered Jul 27 '23 at 19:09

0

The url_path and method are not part of the DefaultStatTags that get extracted automatically.

To make it work, you must add the extraStatTags annotation (sidecar.istio.io/extraStatTags: request_path,request_method) to the deployment (or configure it mesh-wide via mesh config). See the example in the docs.

Also, you should consider upgrading to the newer version of Istio as 1.8 was released in 2020 and not supported anymore.

answered Jul 27 '23 at 19:09

peterj

103
6

You're right but looks like I forgot to mention that I already added you said. That's how I made it work for `method`. My problem is metrics are getting duplicated with request path prefixes but it's not the case for request method. I successfully added method as a label but not request path. – ezileli Jul 30 '23 at 19:25

Customizing Istio metrics results in duplicated metrics

1 Answers1