I am trying to set up SSL on port 443 on an ingressgateway. I can consistently reproduce with a very basic setup. I know it is something I am probably doing wrong but haven't been able to figure it out.
My k8s cluster is running on EKS. k version 1.19
I created a certificate with AWS Certificate Manager for domain api.foo.com and additional names *.api.foo.com
The certificate was created successfully and has ARN arn:aws:acm:us-west-2:<some-numbers>:certificate/<id>
Then I did a vanilla install of istio:
istioctl install --set meshConfig.accessLogFile=/dev/stdout
With version:
client version: 1.7.0
control plane version: 1.7.0
This is my gateway definition:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: foo-gateway
annotations:
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-west-2:<some-numbers>:certificate/<id>"
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
service.beta.kubernetes.io/aws-load-balancer-type: "elb"
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
- port:
number: 443
name: https-443
protocol: HTTP
hosts:
- "*"
Note that port 443 has protocol HTTP, I don't believe that is the problem (since I want to use SSL termination). Also even if I change it to HTTPS, then I get this:
Resource: "networking.istio.io/v1alpha3, Resource=gateways", GroupVersionKind: "networking.istio.io/v1alpha3, Kind=Gateway"
Name: "foo-gateway", Namespace: "default"
for: "foo-gateway.yaml": admission webhook "validation.istio.io" denied the request: configuration is invalid: server must have TLS settings for HTTPS/TLS protocols
But then what would be the tls settings? I need the certificate key to be picked up through the annotation (from AWS CM) not placed in /etc. As an aside, is there a way to do this without ssl termination?
My VirtualService definition is this:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: foo-api
spec:
hosts:
- "*"
gateways:
- foo-gateway
http:
- match:
- uri:
prefix: /users
route:
- destination:
host: https-user-manager
port:
number: 7070
I then k apply -f a super simple REST service called https-user-manager on port 7070. I then find the host name for the load balancer from a k get svc -n istio-system which yields:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-ingressgateway LoadBalancer <cluster-ip> blahblahblah.us-west-2.elb.amazonaws.com 15021:30048/TCP,80:30210/TCP,443:31349/TCP,15443:32587/TCP 32m
I can successfully use http like:
curl http://blahblahblah.us-west-2.elb.amazonaws.com/users and get a valid response
But then if I do this:
curl -vi https://blahblahblah.us-west-2.elb.amazonaws.com/users I get the following:
* Trying <ip>...
* TCP_NODELAY set
* Connected to api.foo.com (<ip>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
* Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
What am I doing wrong? I have seen these https://medium.com/faun/managing-tls-keys-and-certs-in-istio-using-amazons-acm-8ff9a0b99033, Istio-ingressgateway with https - Connection refused, Setting up istio ingressgateway, SSL Error - wrong version number (HTTPS to HTTP), Updating Istio-IngressGateway TLS Cert, https://github.com/kubernetes/ingress-nginx/issues/3556, https://github.com/istio/istio/issues/14264, https://preliminary.istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/, https://preliminary.istio.io/latest/docs/tasks/traffic-management/ingress/ingress-sni-passthrough/, among many others that I don't even remember anymore. Would appreciate any help!
