0

I have an ingress-nginx controller installed though helm in a namespace called nginx

My services run in the default namespace

I have a SecretProviderClass in the nginx namespace, which is referenced from the nginx controller yaml in the nginx namespace.

When I deploy an Ingress resource into the default namespace, and I go into the logs of the controller pod, I see

W0930 13:57:10.224167       7 backend_ssl.go:47] Error obtaining X.509 certificate: no object matching key "default/ingress-tls-csi" in local store

Clearly it is looking in the wrong namespace for the secret.

What is the right way to handle this? I have tried to duplicate the SecretProviderClass in the default namespace but it doesn't seem to be creating the secret. I have also tried to point my Ingress to nginx/ingress-tls-csi instead of just ingress-tls-csi but it complains about changing an immutable field when I try to deploy that even if I delete the Ingress resource first.

The docs I am referencing are https://learn.microsoft.com/en-us/azure/aks/csi-secrets-store-nginx-tls

Conveniently there they are using the same namespace for the services and the controller. I'd prefer not to do that as the controller needs privilege escalation and I don't want to allow that in the namespace my service run in.

Josh
  • 1,648
  • 8
  • 27
  • 58
  • The secret itself must exist in the same namespace as what's trying to use. – jordanm Oct 01 '22 at 01:25
  • What is the point of setting up a SecretProviderClass and attaching it to the Nginx Controller if the controller isn't going to use it to pull the secret? Do I even need to attach it to the controller? – Josh Oct 03 '22 at 13:11
  • The provider class itself is global, but the actual secret is not. – jordanm Oct 03 '22 at 13:30

1 Answers1

0

We solve a similar problem using reflector:

https://github.com/EmberStack/kubernetes-reflector

It is a Kubernetes addon that keeps ConfigMaps and Secrets in sync across namespaces.

Install reflector and add the following annotations to the secret you need to have available in the default namespace:

reflector.v1.k8s.emberstack.com/reflection-allowed: "true"
reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "default"
HiroCereal
  • 550
  • 1
  • 11