After I inject network delay of, say 5s, using add_delay_fault function, the delay is nicely effected for all traffic coming in via the Istio ingress gateway LB. However, if I try to call the same application service (which has delay configured) from a dummy nginx pod in the same cluster, the response is immediate 'without' any delay.
Given that every app pod has the envoy proxy car with it(which is implementing all the Istio rules) I was assuming that even the intra-cluster communication should have same effect. I couldn't find much from the documentation. Would be great if experts here can throw some light on this behaviour.
Here's one way i injected the delay - as a manifest for chaostoolkit.
version: 1.0.0
title: What happens if we abort and delay responses
description: If responses are aborted and delayed, the dependant application should retry and/or timeout requests
tags:
- k8s
- istio
- http
configuration:
ingress_host:
type: env
key: INGRESS_HOST
method:
- type: action
name: delay
provider:
type: python
module: chaosistio.fault.actions
func: add_delay_fault
arguments:
virtual_service_name: newapp
fixed_delay: 5s
routes:
- destination:
host: newapp
subset: v1
percentage: 100
version: networking.istio.io/v1alpha3
ns: default
The result is the same even when i directly changed the virtual service as a k8s manifest as below, i.e, the delay exists if i access through the gateway but no delay when the service is directly called from another pod in the same cluster:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: newapp
spec:
hosts:
- newapp
- "newapp.example.com"
gateways:
- newapp-gateway
http:
- fault:
delay:
percent: 100
fixedDelay: 5s
route:
- destination:
host: newapp
subset: v1
