GKE - Using Google managed certificate (mcrt) for SSL connection to Extensible service proxy (ESP)

Question

I am currently trying to set up multiple Cloud Endpoints with my API services running inside of a GKE cluster. I am using an Ingress to expose the ESP to the internet and I have issued a managed certificate to access the proxy using HTTPS. This is the configuration of my ingress:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: mw-ingress
  annotations:
    networking.gke.io/managed-certificates: mw-cert
    kubernetes.io/ingress.global-static-ip-name: mw-static-ip
spec:
  backend:
    serviceName: frontend-service
    servicePort: 80
  rules:
    - http:
        paths:
          - path: /auth/api/*
            backend:
              serviceName: auth-service
              servicePort: 8083

While this is the deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: auth
  name: auth
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: auth
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: auth
    spec:
      volumes:
        - name: cloud-endpoints-credentials-volume
          secret:
            secretName: cloud-endpoints-secret
      containers:
        - name: auth-service
          image: eu.gcr.io/my-project/auth-service
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8083
              protocol: TCP
        - name: esp
          image: gcr.io/endpoints-release/endpoints-runtime:1
          args: [
            "--backend=127.0.0.1:8083",
            "--http_port=8084",
            "--service=auth-service.endpoints.my-project.cloud.goog",
            "--rollout_strategy=managed",
            "--service_account_key=/etc/nginx/creds/cloudendpoint.json",
            "-z", "healthz"
          ]
          ports:
            - containerPort: 8084
          volumeMounts:
            - name: cloud-endpoints-credentials-volume
              mountPath: /etc/nginx/creds
              readOnly: true

Up to this point everything is working fine.

However I cannot seem to find a way to enable SSL on the ESP. The official documentation says to create a secret from the certificate files. However as Google provisions the certificate I have no idea how to create a secret from it. All of the hints and comments I could find on other sources are using self-signed certificates and/or cert-manager like this: https://github.com/GoogleCloudPlatform/endpoints-samples/issues/52#issuecomment-454387373

They mount a volume containing that secret inside of the deployment. When I just try to add the flag "-ssl_port=443" to the list of arguments on the ESP I obviously get the following error during deployment because the certificate is not there: nginx: [emerg] BIO_new_file("/etc/nginx/ssl/nginx.crt") failed (SSL: error:02000002:system library:OPENSSL_internal:No such file or directory:fopen('/etc/nginx/ssl/nginx.crt','r') error:1100006e:BIO routines:OPENSSL_internal:NO_SUCH_FILE)

Has anybody used managed certificates in combination with the ESP before and has an idea on how to mount the certificate or create a secret?

Answer 1

I ran into the same issue. Solution was to upload my cert as a secret and mount the secret to the esp container in the location it's expecting. According to the documentation it's hard-coded in the esp container to look for the certs at a specific file path and with a specific naming convention.

https://cloud.google.com/endpoints/docs/openapi/specify-proxy-startup-options?hl=tr

  - name: esp
    image: gcr.io/endpoints-release/endpoints-runtime:1
    volumeMounts:
      - mountPath: /etc/nginx/ssl
        name: test-ssl
        readOnly: true
  .
  .
  .
  volumes:
    - name: test-ssl
      projected:
        sources:
        - secret:
            name: test-ssl
            items:
            - key: dev.crt
              path: nginx.crt
            - key: dev.key
              path: nginx.key