I help maintain a web app and everything is fully automated in the deploy pipeline to Kubernetes except our nginx-ingress changes. When an ingress change does come down the pipeline an experienced human will manually vet it and deploy it via helm. Fear is that if a developer makes a change to the ingress incorrectly that it will be promoted live and bring either a portion of or the entire application to a halt.
We'd like the ability to automate this process, but I'm not finding good resources on how to accomplish this. I looked at Linkerd, Istio and Weaveworks Flagger. Flagger seems like it would be exactly what I'm looking for (canary, automated metric checks, automated rollback, even custom checks like bash scripts). But it canaries services, not the ingress itself. My next course of action is going to be recommending writing an entirely custom way of doing this (maybe a CRD or service that does what flagger does but for ingress records, not services) but I wanted to reach out to the internet and see if someone here could recommend a product of any kind that can do what I'm looking to do? Safely canary (or just straight up deploy) an ingress change.
