Istio Traffic Splitting Not Using Weight Properly

Question

I am using the default bookinfo application https://istio.io/docs/examples/bookinfo/ and trying to test split traffic with the reviews service. Kiali is showing the split and everything seems to be configured correctly but its still doing a round robin. If I remove all virtual services and destination rules, the app works as normally expected.

# Source: bookinfo/templates/destination-rule-all.yaml
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: productpage
spec:
  host: productpage
  subsets:
  - name: v1
    labels:
      version: v1
---
# Source: bookinfo/templates/destination-rule-all.yaml
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: ratings
spec:
  host: ratings
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
  - name: v2-mysql
    labels:
      version: v2-mysql
  - name: v2-mysql-vm
    labels:
      version: v2-mysql-vm
---
# Source: bookinfo/templates/destination-rule-all.yaml
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: details
spec:
  host: details
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
---
# Source: bookinfo/templates/destination-rule-all.yaml
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: reviews
spec:
  host: reviews
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
  - name: v3
    labels:
      version: v3
---
# Source: bookinfo/templates/bookinfo-gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: bookinfo-gateway
  chart: bookinfo-0.1.2
  release: bookinfo
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
# Source: bookinfo/templates/bookinfo-gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
  chart: bookinfo-0.1.2
  release: bookinfo
spec:
  hosts:
  - "*"
  gateways:
  - bookinfo-gateway
  http:
  - match:
    - uri:
        exact: /productpage
    - uri:
        prefix: /static
    - uri:
        exact: /login
    - uri:
        exact: /logout
    - uri:
        prefix: /api/v1/products
    route:
    - destination:
        host: productpage
        port:
          number: 9080
---
# Source: bookinfo/templates/virtual-service-all-v1.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: productpage
spec:
  hosts:
  - productpage
  gateways:
  - bookinfo-gateway
  http:
  - route:
    - destination:
        host: productpage
        subset: v1
---
# Source: bookinfo/templates/virtual-service-all-v1.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  gateways:
  - bookinfo-gateway
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 100
    - destination:
        host: reviews
        subset: v2
      weight: 0
    - destination:
        host: reviews
        subset: v3
      weight: 0
---
# Source: bookinfo/templates/virtual-service-all-v1.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: ratings
spec:
  hosts:
  - ratings
  gateways:
  - bookinfo-gateway
  http:
  - route:
    - destination:
        host: ratings
        subset: v1
---
# Source: bookinfo/templates/virtual-service-all-v1.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: details
spec:
  hosts:
  - details
  gateways:
  - bookinfo-gateway
  http:
  - route:
    - destination:
        host: details
        subset: v1

# Source: bookinfo/templates/reviews-v1-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: reviews-v1
  labels:
    chart: bookinfo-0.1.2
    release: bookinfo
    app: reviews
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: reviews
      release: bookinfo
      version: v1
  template:
    metadata:
      labels:
        app: reviews
        release: bookinfo
        version: v1
    spec:
      serviceAccountName: bookinfo-reviews
      containers:
      - name: reviews
        image: myhub/istio/examples-bookinfo-reviews-v1:1.15.0
        imagePullPolicy: IfNotPresent
        env:
        - name: LOG_DIR
          value: "/tmp/logs"
        ports:
        - containerPort: 9080
        volumeMounts:
        - name: tmp
          mountPath: /tmp
        - name: wlp-output
          mountPath: /opt/ibm/wlp/output
      volumes:
      - name: wlp-output
        emptyDir: {}
      - name: tmp
        emptyDir: {}
---
# Source: bookinfo/templates/reviews-v2-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: reviews-v2
  labels:
    chart: bookinfo-0.1.2
    release: bookinfo
    app: reviews
    version: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: reviews
      release: bookinfo
      version: v2
  template:
    metadata:
      labels:
        app: reviews
        release: bookinfo
        version: v2
    spec:
      serviceAccountName: bookinfo-reviews
      containers:
      - name: reviews
        image: myhub/istio/examples-bookinfo-reviews-v2:1.15.0
        imagePullPolicy: IfNotPresent
        env:
        - name: LOG_DIR
          value: "/tmp/logs"
        ports:
        - containerPort: 9080
        volumeMounts:
        - name: tmp
          mountPath: /tmp
        - name: wlp-output
          mountPath: /opt/ibm/wlp/output
      volumes:
      - name: wlp-output
        emptyDir: {}
      - name: tmp
        emptyDir: {}
---
# Source: bookinfo/templates/reviews-v3-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  metadata:
  name: reviews-v3
  labels:
    chart: bookinfo-0.1.2
    release: bookinfo
    app: reviews
    version: v3
spec:
  replicas: 1
  selector:
    matchLabels:
      app: reviews
      release: bookinfo
      version: v3
  template:
    metadata:
      labels:
        app: reviews
        release: bookinfo
        version: v3
    spec:
      serviceAccountName: bookinfo-reviews
      containers:
      - name: ratings
        image: myhub/istio/examples-bookinfo-reviews-v3:1.15.0
        imagePullPolicy: IfNotPresent
        env:
        - name: LOG_DIR
          value: "/tmp/logs"
        ports:
        - containerPort: 9080
        volumeMounts:
        - name: tmp
          mountPath: /tmp
        - name: wlp-output
          mountPath: /opt/ibm/wlp/output
      volumes:
      - name: wlp-output
        emptyDir: {}
      - name: tmp
        emptyDir: {}
---

Environment

kind v0.7.0 go1.13.6 linux/amd64

K8s v1.18.1 v1.17.0

score 1 · Answer 1 · answered May 13 '20 at 08:11

I tried to reproduce your problem on gke with istio 1.5.2 and everything works fine.

I followed istio bookinfo documentation

kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml

As mentioned here

Apply default destination rules

Before you can use Istio to control the Bookinfo version routing, you need to define the available versions, called subsets, in destination rules.

Run the following command to create default destination rules for the Bookinfo services:

If you did not enable mutual TLS, execute this command:

Choose this option if you are new to Istio and are using the demo configuration profile.

$ kubectl apply -f samples/bookinfo/networking/destination-rule-all.yaml

If you did enable mutual TLS, execute this command:

$ kubectl apply -f samples/bookinfo/networking/destination-rule-all-mtls.yaml

Wait a few seconds for the destination rules to propagate.

You can display the destination rules with the following command:

$ kubectl get destinationrules -o yaml

Apply your virtual service for reviews

Examples

Traffic only for subset v1.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1

Traffic only for subset v2.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v2

50/50 weight traffic for subset v2 and v3.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v2
      weight: 50
    - destination:
        host: reviews
        subset: v3
      weight: 50

100/0/0 weight traffic for subset v1,v2 and v3.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v2
      weight: 100
    - destination:
        host: reviews
        subset: v3
      weight: 0
    - destination:
        host: reviews
        subset: v1
      weight: 0

With virtual service only for subset v1 Kiali shows traffic goes only to v1

Istio productpage doesn't show the stars, so it's review v1.

There is documentation about weight-based routing for reviews.

So strange, I updated my post with pics from Kiali. It's showing everything is correctly configured, but when I refresh the bookinfo app I get red, black and no stars. I even tried deleting all virtual services and destination rules and the app still works normally. Seems to be traffic isnt be forced through the proxy — CodyK, May 13 '20 at 17:06
That's weird. Is your istio deployed in istio-system namespace? What is your istio version? — Jakub, May 14 '20 at 06:34
Very strange. Version 1.5.1 and yes its in the istio-system namespace but bookinfo is in its own namespace but ingress is working. I am also running my environment locally via Kind, so could be a defect with the environment. — CodyK, May 14 '20 at 17:55

Istio Traffic Splitting Not Using Weight Properly

1 Answers1

Apply default destination rules

Apply your virtual service for reviews