4

I am working with a Kubernetes cluster running in Azure Container Services. We use SSL Interception, which results in any kubectl command returning this error:

Unable to connect to the server: x509: certificate signed by unknown authority

I'm working on macOS 10.12.6

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.1", GitCommit:"f38e43b221d08850172a9a4ea785a86a3ffa3b3a", GitTreeState:"clean", BuildDate:"2017-10-11T23:27:35Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.9", GitCommit:"19fe91923d584c30bd6db5c5a21e9f0d5f742de8", GitTreeState:"clean", BuildDate:"2017-10-19T16:55:06Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

Here is the verbose output for running kubectl version

$ kubectl version --v=10
I1128 10:27:28.721914   16346 loader.go:357] Config loaded from file /Users/foo/.kube/config
I1128 10:27:28.726719   16346 round_trippers.go:417] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.8.1 (darwin/amd64) kubernetes/f38e43b" https://foo.westus2.cloudapp.azure.com/version
I1128 10:27:29.046962   16346 round_trippers.go:436] GET https://foo.westus2.cloudapp.azure.com/version  in 320 milliseconds
I1128 10:27:29.046989   16346 round_trippers.go:442] Response Headers:
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.1", GitCommit:"f38e43b221d08850172a9a4ea785a86a3ffa3b3a", GitTreeState:"clean", BuildDate:"2017-10-11T23:27:35Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"darwin/amd64"}
I1128 10:27:29.047291   16346 helpers.go:225] Connection error: Get https://foo.westus2.cloudapp.azure.com/version: x509: certificate signed by unknown authority
F1128 10:27:29.047310   16346 helpers.go:120] Unable to connect to the server: x509: certificate signed by unknown authority

Things I have tried:

  • Setting --insecure-skip-tls-verify=true results in another error Unable to connect to the server: EOF
  • Setting https_proxy to a proxy that is not under SSL Interception works correctly
  • Imported and trusted the ca from the apiserver
Christopher Stevenson
  • 41
  • 2

1 Answers1

0

kubectl performs its own verification of the cluster's certificate outside of the OS' trusted certificate authorities, using the certificate authority defined in the certificate-authority-data field of the Kubeconfig file.

It is likely that the device performing the interception is a) replacing the cluster's certificate with its own, signed by a difference CA and b) unable to verify the cluster's certificate, which is signed by a different CA.

Depending on how the cluster is built, you may be able to issue certificates based on a pre-existing CA trusted by the interception device instead of a separate CA for the cluster. This is far beyond the scope of a StackOverflow answer; Kubernetes Up and Running and Kubernetes the Hard Way are good starting points, and you may need to read Kubernetes source code to get all the Key Usages of the certificates right.

It should be noted that SSL Interception is considered a bad practice (though it is widespread in some industries such as banking). TLS 1.3 is designed to prevent it entirely through the exclusive use of Perfect Forward Secrecy.

Dharma Bellamkonda
  • 704
  • 4
  • 11