3

I have a following use case:

  1. Our customers frequently release new services on their K8s clusters. These new services are reachable from the outside world through a load balancing and Ingress to dynamically configure this load balancing once a service is deployed. This makes it really easy for the development teams of our customers because they don’t have to wait until somebody configures a load balancing manually. They can just create their own Ingress resource next to their service deployment and the service will be reachable.

  2. A customer asked if we can also enable that each of its services can have its own subdomain automatically. So once a new application is deployed it, it should be available as a subdomain of the cluster domain (e.g. https://helloworld.cyvh5.k8s.ginger.aws.gigantic.io) as well as at their own subdomain (e.g.. helloworld.awesome-customer.com).

I have found this resource as a starting point.

My questions are:

  1. Can I achieve the customer subdomain dynamic binding in some other (better) way?

  2. What are the possible limitations / pitfalls for the suggested solution?

Thanks!

Angel Todorov
  • 1,443
  • 2
  • 19
  • 37

1 Answers1

3

Yeah for 1 ingress sounds great.

For 2 it sounds to me like you just need wildcard DNS pointing at the ingress controller. The wildcard DNS entry should say that *.domain.com should point to the ingress controller's external IP. Then host-based Ingress rules/resources can be deployed and traffic can be routed to the appropropriate Service based on the host specified in the request. So it doesn't matter what is in the wildcard part of the DNS of a request insofar as 'a.b.domain.com' will go to the ingress controller and it will then depend on what rules are in the Ingress resources as to where it ends up.

This won't be 'automatic' in the sense that the customer will have to deploy an Ingress rule or two if they want the service exposed on two hosts. But if the customer is happy with deploying Ingress resources then they should be happy with this too.

I don't think you need anything more dynamic because in 'helloworld.awesome-customer.com' it seems 'helloworld' is the service so that fills out your host so there's no need for a wildcard in the Ingress rule itself. What would be more dynamic and more like the example you point to is if they were asking for 'v1.helloworld.awesome-customer.com' and 'v2.helloworld.awesome-customer.com' and for both to be covered by one Ingress entry containing a wildcard (rather than two entries, one per version). But it seems they are not asking for that.

This is how I see the customer domain part anyway. I am not exactly sure what you mean about the cluster domain part - for that I'd need to better understand how that is accessed. Presumably it is again wildcard DNS pointing at something doing routing but I'm not as sure about what is doing the routing there. If the point is that you want to achieve this then it could just be that it's another wildcard DNS entry pointed at the same ingress controller with additional Ingress resources deployed.

Ryan Dawson
  • 11,832
  • 5
  • 38
  • 61
  • Thanks for your answer. To make sure, I have got you right: when a customer creates an Ingress resource in the scope of #1, they will also has to add the rule, which will allow later their service to be available via designated sub domain e.g. `helloworld.awesome-customer.com`, hence to comply with #2 from the reqs? And all this can be achieved by add Ingress DNS wildcard? – Angel Todorov Dec 21 '18 at 20:17
  • I mean that the wildcard DNS and the Ingress are separate. The wildcard DNS entry points at the ingress controller running in the cluster and that's a one-off setup job (perhaps you would do that bit). Then host entries under that domain can be used in Ingress resources that the customer deploys. – Ryan Dawson Dec 21 '18 at 21:26
  • An example of using wildcard DNS and Ingress that I've worked on is https://github.com/Activiti/activiti-cloud-charts/blob/master/activiti-cloud-full-example/README.md if it helps. – Ryan Dawson Dec 21 '18 at 21:28
  • Related question on this in case the explanation there helps - https://stackoverflow.com/questions/53865013/whats-the-exactly-flow-chart-of-an-outside-request-comes-into-k8s-pod-via-ingre – Ryan Dawson Dec 21 '18 at 22:38
  • 1
    thanks again. Let me see if I got it now: 1) I need a wildcard DNS pointing to the Ingress controller; 2) Into Ingress yml file I will add all subdomains, each one pointing to an existing service within the cluster. In that case each time a brand new service has been deployed the ingress file has to be updated with a new subdomain name. Analogically - in case service has been removed its subdomain has to be removed as well. Basically, I should follow https://stackoverflow.com/questions/50261504/dynamically-adding-removing-named-hosts-from-k8s-ingress. Am I right? – Angel Todorov Dec 22 '18 at 08:50
  • 1
    Yep, that explains it very well. – Ryan Dawson Dec 22 '18 at 09:49
  • And to close the subject - is there a way to automate the process, meaning once a new service has been released, the Ingress resource file to be updated accordingly. Vise versa - in case the service is terminated, its section from Ingress resource file to be removed? – Angel Todorov Dec 22 '18 at 15:24
  • 1
    Yes Jenkins-X expose contoller will auto generate ingress from services. But I find some apps need a lot of customisation in the ingress, especially UIs (mostly for path rewriting or the app itself needing to know the host). So I prefer to treat the ingress as source code as the official kubernetes helm charts do. – Ryan Dawson Dec 22 '18 at 17:19
  • I should say it is worth keeping an eye on the externalDNS project if you're interested in automating the DNS part, though that project is currently in incubator. – Ryan Dawson Dec 23 '18 at 18:02