Kubernetes dashboard: TLS handshake error

Question

I have an EKS 1.18 cluster. When I tried to deploy a k8s dashboard, it's failing with the error below.

Also, my dashboard svc uses a loadBalancer.

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  annotations:
    external-dns.alpha.kubernetes.io/hostname: "test.xxx.net"
    service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  type: LoadBalancer
  ports:
    - port: 443
      targetPort: 8443
  selector:
    k8s-app: kubernetes-dashboard

Pls let me know what does the TLS handshake error mean? What should I do to fix this error?

logs:

2021/03/18 22:03:08 http: TLS handshake error from xx.xxx.x.x:8279: EOF
2021/03/18 22:03:08 http: TLS handshake error from xx.xxx.x.x:34935: EOF
2021/03/18 22:03:08 http: TLS handshake error from xx.xxx.x.x:24437: EOF
2021/03/18 22:03:08 http: TLS handshake error from xx.xxx.x.x:64552: EOF
2021/03/18 22:03:10 http: TLS handshake error from xx.xxx.x.x:5481: EOF

code:

https://github.com/kubernetes/dashboard/releases/tag/v2.0.3

https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.3/aio/deploy/recommended.yaml

 kubectl version
+ kubectl version Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.1", GitCommit:"d647ddbd755faf07169599a625faf302ffc34458", GitTreeState:"clean", BuildDate:"2019-10-02T23:49:20Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.9-eks-d1db3c", GitCommit:"d1db3c46e55f95d6a7d3e5578689371318f95ff9", GitTreeState:"clean", BuildDate:"2020-10-20T22:18:07Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

Your ingress presumably does not understand the dashboard terminates tls itself and tries to send naked http requests there. So, whatever you use to ingress/load balance traffic to the app - should be configured to connect via tls. — zerkms, Mar 19 '21 at 02:55
How exactly are you accessing your dashboard ? Using load balancer's IP address or some domain ? Have you also configured any ingress resource ? — mario, Mar 19 '21 at 20:23
"there isn't any ingress configured" --- well, whoever connects to it does http without tls — zerkms, Mar 22 '21 at 20:18

score 0 · Answer 1 · answered Apr 07 '21 at 13:18

Explanation:

zerkms already shed some light on the cause of your issue in his comment:

Your ingress presumably does not understand the dashboard terminates tls itself and tries to send naked http requests there. So, whatever you use to ingress/load balance traffic to the app - should be configured to connect via tls. – zerkms Mar 19 at 2:55

You also told us that there isn't any ingress configured but you use for connection the domain name, pointing to your LoadBalancer's IP. That's fine but keep in mind that when you create a Service of LoadBalancer type on your EKS cluster, by default Classic Load Balancer is created. It works on layer 7 of the OSI model so it recognizes the https traffic, terminates TLS connection and then sends to your backend pods naked http request. As zerkms already explained, your backend is not prepared to handle such connection as it terminates TLS itself.

Solution:

As I already mentioned ,by default, when you create your LoadBalancer service, Classic Load Balancer is created. However you can change this default behaviour by adding to your Service the following annotation as mentioned here:

service.beta.kubernetes.io/aws-load-balancer-type: nlb

As Network Load Balancer operates on layer 4 of the OSI model, it simply passes TCP packets to your backend pods without inspecting their content, terminating TLS etc and expects https traffic.

Alternatively you may set up some ingress controller which is configured to support SSL-passthrough like ngix-ingress as the AWS's ALB unfortunatelly doesn't support it.

Kubernetes dashboard: TLS handshake error

1 Answers1

Explanation:

Solution: