Simple front end application on EKS via AppMesh

Question

Kindly ask you to help to find out the problem with my configuration. It was done on the scope of AWS WorkShop example just rewrite on another HTTP container. Right now, after implementation of this, everything is up, but when going on NLB getting "no healthy upstream".

Have checked the logs, and see only 503 errors on my Gateway Ingress. Requests are not coming to my pod at all. Where I made mistake in my configuration?

apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualGateway
metadata:
  name: ingress-gw
  namespace: shared
spec:
  namespaceSelector:
    matchLabels:
      gateway: shared-gw
  podSelector:
    matchLabels:
      app: ingress-gw
  listeners:
    - portMapping:
        port: 8088
        protocol: http
  logging:
    accessLog:
      file:
        path: /dev/stdout
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpd-echo-deployment
  namespace: shared
  labels:
    app: httpd-echo1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpd-echo1
  template:
    metadata:
      labels:
        app: httpd-echo1
      annotations:
        appmesh.k8s.aws/mesh: shared-mesh
    spec:
      containers:
      - name: httpd
        image: hashicorp/http-echo
        args:
          - "-text=test"
        ports:
        - containerPort: 5678
---
apiVersion: v1
kind: Service
metadata:
  namespace: shared
  name: httpd-echo-service
  labels:
    app: httpd-echo1
spec:
  ports:
    - name: "http"
      port: 5678
      targetPort: 5678
  selector:
    app: httpd-echo1
---
apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualNode
metadata:
  name: shared-virtual-node-1
  namespace: shared
spec:
  podSelector:
    matchLabels:
      app: httpd-echo1
  listeners:
    - portMapping:
        port: 5678
        protocol: http
      healthCheck:
        protocol: http
        path: '/'
        healthyThreshold: 5
        unhealthyThreshold: 5
        timeoutMillis: 2000
        intervalMillis: 5000
  serviceDiscovery:
    dns:
      hostname: httpd-echo1.test.com
  logging:
    accessLog:
      file:
        path: /dev/stdout
---
apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualService
metadata:
  name: shared-virtual-service-1
  namespace: shared
spec:
  awsName: httpd-echo1.test.com
  provider:
    virtualNode:
      virtualNodeRef:
        name: shared-virtual-node-1
---
apiVersion: appmesh.k8s.aws/v1beta2
kind: GatewayRoute
metadata:
  name: shared-gw-route-1
  namespace: shared
spec:
  httpRoute:
    match:
      prefix: "/"
    action:
      target:
        virtualService:
          virtualServiceRef:
            name: shared-virtual-service-1
---
apiVersion: v1
kind: Service
metadata:
  name: ingress-gw
  namespace: shared
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-load-balancer-subnets : subnet-1,subnet-2,subnet-3
    service.beta.kubernetes.io/aws-load-balancer-internal: "false"
spec:
  type: LoadBalancer
  ports:
    - port: 80
      targetPort: 8088
      name: http
  selector:
    app: ingress-gw
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ingress-gw
  namespace: shared
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-gw
  template:
    metadata:
      labels:
        app: ingress-gw
    spec:
      containers:
        - name: envoy
          image: 422531588944.dkr.ecr.eu-south-1.amazonaws.com/aws-appmesh-envoy:v1.16.1.1-prod
          ports:
            - containerPort: 8088

Example which I have tried to use : https://github.com/aws-containers/eks-app-mesh-polyglot-demo/tree/cf15e0d8e10c019d332f5378d132a8d620131df8/deployment

score 2 · Accepted Answer · answered Apr 09 '21 at 18:22

I tried to reproduce the same at my side and it worked fine. There are couple of configuration changes I did to the above yaml.

Added the gateway label “gateway: shared-gw“ to the VirtualGateway. Make sure that you have this label in the namespace as well.
Corrected the dns hostname. This should be your application clusterIp service name serviceDiscovery: dns: hostname: httpd-echo1.shared.svc.cluster.local

Also, ensure that your Laodbalancer is Active and the target group listener for this LB is showing healthy status

I am adding the updated yaml below. You can try this and see if it works.

---
apiVersion: v1
kind: Namespace
metadata:
  name: shared
  labels:
    mesh: shared-mesh
    gateway: ingress-gw
    appmesh.k8s.aws/sidecarInjectorWebhook: enabled
---
apiVersion: appmesh.k8s.aws/v1beta2
kind: Mesh
metadata:
  name: shared-mesh
spec:
  namespaceSelector:
    matchLabels:
      mesh: shared-mesh
---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpd-echo1
  namespace: shared
  labels:
    app: httpd-echo1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpd-echo1
  template:
    metadata:
      labels:
        app: httpd-echo1
      annotations:
        appmesh.k8s.aws/mesh: shared-mesh
    spec:
      containers:
      - name: httpd
        image: hashicorp/http-echo
        args:
          - "-text=test"
        ports:
        - containerPort: 5678
---
apiVersion: v1
kind: Service
metadata:
  namespace: shared
  name: httpd-echo1
  labels:
    app: httpd-echo1
spec:
  ports:
    - name: "http"
      port: 5678
      targetPort: 5678
  selector:
    app: httpd-echo1
---
apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualNode
metadata:
  name: shared-virtual-node-1
  namespace: shared
spec:
  podSelector:
    matchLabels:
      app: httpd-echo1
  listeners:
    - portMapping:
        port: 5678
        protocol: http
      healthCheck:
        protocol: http
        path: '/'
        healthyThreshold: 5
        unhealthyThreshold: 5
        timeoutMillis: 2000
        intervalMillis: 5000
  serviceDiscovery:
    dns:
      hostname: httpd-echo1.shared.svc.cluster.local
  logging:
    accessLog:
      file:
        path: /dev/stdout
---
apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualService
metadata:
  name: shared-virtual-service-1
  namespace: shared
spec:
  awsName: httpd-echo1.shared.svc.cluster.local
  provider:
    virtualNode:
      virtualNodeRef:
        name: shared-virtual-node-1
---

apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualGateway
metadata:
  name: ingress-gw
  namespace: shared
spec:
  namespaceSelector:
    matchLabels:
      gateway: ingress-gw
  podSelector:
    matchLabels:
      app: ingress-gw
  listeners:
    - portMapping:
        port: 8088
        protocol: http
  logging:
    accessLog:
      file:
        path: /dev/stdout
---
apiVersion: v1
kind: Service
metadata:
  name: ingress-gw
  namespace: shared
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
spec:
  type: LoadBalancer
  ports:
    - port: 80
      targetPort: 8088
      name: http
  selector:
    app: ingress-gw
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ingress-gw
  namespace: shared
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-gw
  template:
    metadata:
      labels:
        app: ingress-gw
    spec:
      containers:
        - name: envoy
          image: 422531588944.dkr.ecr.eu-south-1.amazonaws.com/aws-appmesh-envoy:v1.16.1.1-prod
          ports:
            - containerPort: 8088
---
apiVersion: appmesh.k8s.aws/v1beta2
kind: GatewayRoute
metadata:
  name: shared-gw-route-1
  namespace: shared
spec:
  httpRoute:
    match:
      prefix: "/"
    action:
      target:
        virtualService:
          virtualServiceRef:
            name: shared-virtual-service-1
---

Thank you a lot! It's working for me! just maybe if you know and can help with two questions : 1 - is it possible to use some custom domain? not local one? and 2 - how to route on few services via gateway route based on dns? cause always receiving default one — Manish Iarhovich, Apr 10 '21 at 12:38
1. You can use AWS CloudMap for service-discovery. With Cloud Map, you can define custom names for your application resources, and it maintains the updated location of these dynamically changing resources. https://github.com/aws/aws-app-mesh-examples/tree/main/walkthroughs/howto-k8s-cloudmap 2.If I understand your question correctly, you want to use VirtualGateway to route to few services. You can do that by having multiple gateway routes like in this example https://github.com/aws/aws-app-mesh-examples/blob/main/walkthroughs/howto-k8s-ingress-gateway/v1beta2/manifest.yaml.template — praseeda sathaye, Apr 12 '21 at 15:52

Simple front end application on EKS via AppMesh

1 Answers1