Istio strips header when added via EnvoyFilter

Question

I'm trying to add a specific "x-envoy..." header on the inbound request to a specific workload. More precisely it's the x-envoy-max-retries header.

I do it like this:

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: strip-filter
  namespace: default
spec:
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: SIDECAR_INBOUND
      listener:
        filterChain:
          filter:
            name: envoy.filters.network.http_connection_manager
            subFilter:
              name: "envoy.filters.http.router"
    patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.lua
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
          inlineCode: |
            function envoy_on_request(request_handle)
              request_handle:headers():add("test-header", "1")
              request_handle:headers():add("x-envoy-max-retries", "0")
              request_handle:headers():add("test-header", "2")
            end
  workloadSelector:
    labels:
      app: test-app

The test-header is added with it's two values successfully - it can be observed from logs of the sidecar proxy and manually by logging in the workload as well. But the x-envoy-max-retries header is missing (and doesn't get applied).

I observe this when making internal calls (using the fully-qualified name of the workload from another workload), but also from external calls (via VirtualService).

Any ideas how to fix this? I don't want to use VirtualServices and their retry properties for disabling retries, so virtual services are a no-go for me.