Istio: Can I add randomly generated unique value as a header to every request before it reaches my application

Question

I have a RESTful service within a spring boot application. This spring boot app is deployed inside a kubernetes cluser and we have Istio as a service mesh attached to the sidecar of each container pod in the cluster. Every request to my service first hits the service mesh i.e Istio and then gets routed accordingly.

I need to put a validation for a request header and if that header is not present then randomly generate a unique value and set it as a header to the request. I know that there is Headers.HeaderOperations which i can use in the destination rule but how can i generate a unique value every time the header is missing? I dont want to write the logic inside my application as this is a general rule to apply for all the applications inside the cluster

What do you actually need the header for? Some of the [distributed tracing headers](https://istio.io/docs/tasks/observability/distributed-tracing/overview/#trace-context-propagation) have essentially this property (`X-Request-Id:` is unique across an entire end-to-end transaction; `X-B3-SpanId:` will be unique per request). — David Maze, Jan 24 '20 at 00:49
Its a request id which needs to be unique. If the client fails to pass it along i want to add it from my side. Is this x-request-id a known property at istio which I can use it directly — codeninja, Jan 24 '20 at 13:17
Yes the request id headers needs to be unique, but what for? Are You planning to use unique request id headers for tracing paths taken by the requests? — Piotr Malec, Jan 24 '20 at 14:38
Yes for tracing and troubleshooting. Regardless of its usage, i want to know if there is a way to generate a unique id and assign it to the request in case if the request-id header is missing. I want to do it at Istio configuration or using a destination rule — codeninja, Jan 24 '20 at 16:38

score 2 · Answer 1 · answered Dec 18 '20 at 01:59

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: enable-envoy-xrequestid-in-response
  namespace: istio-system
spec:
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: "envoy.http_connection_manager"
    patch:
      operation: MERGE
      value:
        typed_config:
          "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
          always_set_request_id_in_response: true

score 1 · Answer 2 · answered Jan 27 '20 at 17:20

There is important information that needs to be said in this subject. And it looks to me like You are trying to make a workaround tracing for an applications that does not forward/propagate headers in Your cluster. So I am going to mention few problems that can be encountered with this solution (just in case).

As mentioned in answer from Yuri G. You can configure unique x-request-id headers but they will not be very useful in terms of tracing if the requests are passing trough applications that do not propagate those x-request-id headers.

This is because tracing entire request paths needs to have unique x-request-id though out its entire trace. If the x-request-id value is different in various parts of the path the request takes, how are We going to put together the entire trace path?

In a scenario where two requests are received in application pod at the same time even if they had unique x-request-id headers, only application is able to tell which inbound request matches with which outbound connection. One of the requests could take longer to process and without forwarded trace header we can't tell which one is which.

Anyway for applications that do support forwarding/propagating x-request-id headers I suggest following guide from istio documentation.

Hope it helps.

score 1 · Answer 3 · answered Apr 13 '21 at 07:10

The answer above works well! I have updated it for the latest istio (filter name is in full):

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: enable-envoy-xrequestid-in-response
  namespace: istio-system
spec:
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: MERGE
      value:
        typed_config:
          "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
          always_set_request_id_in_response: true

score 0 · Answer 4 · answered Jan 24 '20 at 17:58

From reading the documentation of istio and envoy it seems like this is not supported by istio/envoy out of the box. As a workaround you have 2 options

Option 1: To set the x-envoy-force-trace header in virtual service

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews-route
spec:
  hosts:
  - reviews.prod.svc.cluster.local
  http:
  - headers:
      request:
        set:
          x-envoy-force-trace: true

It will generate a header x-request-id if it is missing. But it seems like abuse of tracing mechanism.

Option 2: To use consistentHash balancing based on header, e.g:

 apiVersion: networking.istio.io/v1alpha3
 kind: DestinationRule
 metadata:
   name: bookinfo-ratings
 spec:
   host: ratings.prod.svc.cluster.local
   trafficPolicy:
     loadBalancer:
       consistentHash:
         httpHeaderName:
           name: x-custom-request-id

It will generate the header x-custom-request-id for any request that doesn't have this header. In this case the requests with same x-custom-request-id value will go always to the same pod that can cause uneven balancing.

Istio: Can I add randomly generated unique value as a header to every request before it reaches my application

4 Answers4