404 Not Found when migrating from ingress to istio

Question

I'm trying to migrate from ingress to istio gateway + virtual service routing, but I keep receiving a 404 Not Found error.

The only link that the app should be accessed to is using my-todos.com, configured locally.

What am I missing here?

Note: the ingress controller works just fine. Initially, todo-lb.default.svc.cluster.local in the istio.yaml file was just set to todo-lb, representing the configured load balancer, still with no success.

Here is the ingress.yaml file (to migrate from):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: todo-ingress
spec:
  rules:
    - host: my-todos.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: todo-lb
                port:
                  number: 3001
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: {{ .Values.api.apiName }}
                port:
                  number: {{ .Values.api.apiPort }}

Here is the istio.yaml file (to migrate TO):

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: todo-istio-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - my-todos.com
    # - "*"
    tls:
      httpsRedirect: true
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: tls-secret
    hosts:
    - my-todos.com
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: todo-lb
spec:
  hosts:
  - my-todos.com
  # - "*"
  gateways:
  - todo-istio-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: todo-lb.default.svc.cluster.local
        port:
          number: 3001
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: todo-api
spec:
  hosts:
  - my-todos.com
  # - "*"
  gateways:
  - todo-istio-gateway
  http:
  - match:
    - uri:
        prefix: /api
    route:
    - destination:
        host: {{ .Values.api.apiName }}
        port:
          number: {{ .Values.api.apiPort }}

score 3 · Answer 1 · answered Jan 27 '21 at 06:40

From what I see you've wrong gateway configuration in your virtual service, that's why it might not work.

If gateway is not in the same namespace as virtual service, you have to specify that in virtual service

Check the spec.gateways section

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo-Mongo
spec:
  gateways:
  - some-config-namespace/my-gateway # can omit the namespace if gateway is in same
                                       namespace as virtual service.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: my-gateway
  namespace: some-config-namespace

There is related istio documentation about that.

So please move your todo-istio-gateway to default namespace.

or use

gateways:
  - istio-system/todo-istio-gateway

Few things to check if that won't help:

Is your app deployed in default namespace?
Is your app injected?

Hello Jakub! Yes, the app is deployed in the default namespace, and it has ```istio-injection=enabled``` — gusti, Jan 27 '21 at 16:16
@DragoșBocancea Could you please add your deployment and service dependencies? Both http and https doesn't work? Also as Malathi mentioned, is there anything in the istio ingress gateway logs? — Jakub, Jan 28 '21 at 08:29

Malathi · Accepted Answer · 2021-01-28T06:41:30.373

In addition to @Jakub answer, there can be one more reason you can get 404 error. Your current ingress rules in virtual service looks like this:

Hostname	Path	Route
my-todos.com	/	Forward to todo-lb.default.svc.cluster.local
my-todos.com	/api	Forward to {{ .Values.api.apiName }}

In Istio, each of the above is an ingress rule. If in Istio ingress-gateway, the rules are added in the above order, then all the URL path including the prefix /api gets routed to the first service i.e. todo-lb.default. It is better to create a single Virtual service like this and then see if the routing works as expected.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: todo-api
spec:
  hosts:
  - my-todos.com
  gateways:
  - <namespace>/todo-istio-gateway
  http:
  - match:
    - uri:
        prefix: /api
    route:
    - destination:
        host: {{ .Values.api.apiName }}
        port:
          number: {{ .Values.api.apiPort }}
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: todo-lb.default
        port:
          number: 3001

I have deleted the second Virtual Service that included the ```/api``` prefix, and also added ```gateways: - istio-system/todo-istio-gateway ``` as suggested by Jakub, but with no success. — gusti, Jan 27 '21 at 16:19
May be you can check the logs of istio-ingressgateway pod in `istio-system` namespace? — Malathi, Jan 28 '21 at 06:44

score 1 · Answer 3 · answered Jul 05 '22 at 21:00

Problem might be with your Gateway api:

404 errors occur when multiple gateways configured with same TLS certificate Configuring more than one gateway using the same TLS certificate will cause browsers that leverage HTTP/2 connection reuse (i.e., most browsers) to produce 404 errors when accessing a second host after a connection to another host has already been established.

For example, let’s say you have 2 hosts that share the same TLS certificate like this:

Wildcard certificate .test.com installed in istio-ingressgateway Gateway configuration gw1 with host service1.test.com, selector istio: ingressgateway, and TLS using gateway’s mounted (wildcard) certificate Gateway configuration gw2 with host service2.test.com, selector istio: ingressgateway, and TLS using gateway’s mounted (wildcard) certificate VirtualService configuration vs1 with host service1.test.com and gateway gw1 VirtualService configuration vs2 with host service2.test.com and gateway gw2 Since both gateways are served by the same workload (i.e., selector istio: ingressgateway) requests to both services (service1.test.com and service2.test.com) will resolve to the same IP. If service1.test.com is accessed first, it will return the wildcard certificate (.test.com) indicating that connections to service2.test.com can use the same certificate. Browsers like Chrome and Firefox will consequently reuse the existing connection for requests to service2.test.com. Since the gateway (gw1) has no route for service2.test.com, it will then return a 404 (Not Found) response.

You can avoid this problem by configuring a single wildcard Gateway, instead of two (gw1 and gw2). Then, simply bind both VirtualServices to it like this:

Gateway configuration gw with host *.test.com, selector istio: ingressgateway, and TLS using gateway’s mounted (wildcard) certificate VirtualService configuration vs1 with host service1.test.com and gateway gw VirtualService configuration vs2 with host service2.test.com and gateway

checkout this link:

https://istio.io/latest/docs/ops/common-problems/network-issues/#404-errors-occur-when-multiple-gateways-configured-with-same-tls-certificate

My solution is using multi ingressgateway for hosts, and different host resolve to different ip address. — lambda, Mar 02 '23 at 02:55

404 Not Found when migrating from ingress to istio

3 Answers3