0

I have two kubernetes cluster setup with kubeadm and im using haproxy to redirect and load balance traffic to the different clusters. Now I want to redirect the requests to the respective api server of each cluster. Therefore, I need to decrypt the ssl requests, read the "Host" HTTP-Header and encrypt the traffic again. My example haproxy config file looks like this:

frontend k8s-api-server
        bind *:6443 ssl crt /root/k8s/ssl/apiserver.pem
        mode http
        default_backend k8s-prod-master-api-server

backend k8s-prod-master-api-server
        mode http
        option forwardfor
        server master 10.0.0.2:6443 ssl ca-file /root/k8s/ssl/ca.crt

If I now access the api server via kubectl, I get the following errors:

kubectl get pods
error: the server doesn't have a resource type "pods"
kubectl get nodes
error: the server doesn't have a resource type "nodes"

I think im using the wrong certificates for decryption and encryption. Do I need to use the apiserver.crt , apiserver.key and ca.crt files in the directory /etc/kubernetes/pki ?

Hector Lorenzo
  • 141
  • 3
  • 11
  • Your setup probably entails authenticating with your Kubernetes API server via client certificates; when your haproxy reinitiates the connection it is not doing so with the client key and certificate on your local machine, and it's likely making an unauthenticated request. As such, it probably doesn't have permission to know about the `pod` and `node` resources, and maybe that's the problem you're seeing? – Amit Kumar Gupta May 16 '20 at 18:45
  • 1
    I don't know if haproxy supports this, but an alternative is to proxy at L4 but reading the SNI header and forwarding traffic that way. This way, you don't need to read any HTTP headers, and thus you don't need to decrypt and re-encrypt the traffic. NGINX supports this, I wouldn't be surprised if haproxy did too. – Amit Kumar Gupta May 16 '20 at 18:47
  • I got it working using SNI as mentioned by @AmitKumarGupta in his last comment – Hector Lorenzo May 16 '20 at 22:18
  • I'll make it a proper answer for others to find more easily – Amit Kumar Gupta May 17 '20 at 00:17

1 Answers1

2

Your setup probably entails authenticating with your Kubernetes API server via client certificates; when your HAProxy reinitiates the connection it is not doing so with the client key and certificate on your local machine, and it's likely making an unauthenticated request. As such, it probably doesn't have permission to know about the pod and node resources.

An alternative is to proxy at L4 by reading the SNI header and forwarding traffic that way. This way, you don't need to read any HTTP headers, and thus you don't need to decrypt and re-encrypt the traffic. This is possible to do with HAProxy.

Amit Kumar Gupta
  • 17,184
  • 7
  • 46
  • 64