we have a problem with designing our Kubernetes infrastructure and it would be a great help if I can have your opinion on the subject. based on the following Diagram, we have a traditional infrastructure with our services implemented in a simple way; our VMs have some services on them and we are using them in production; as you can see, if "Service A" wants to talk to "Service B"(for an API call for example) on a different VM, the request goes to our UTM and we allow that connection with a rule; something like: "deny all; allow from 192.168.1.10 to 192.168.1.20" (demonstrated with the red line);
it's a bit more complex with Kubernetes; my services are randomly distributed across worker nodes(and we don't want to you node affinity), and our service (for example "Service X") can be anywhere on any node; the problem is, if we want to call some of our traditional services outside of the cluster, we don't want to grant access from all of our worker nodes to all of our backends, just because we don't know where our service will be(and assume we have more than 100 backend and more than 100 worker nodes); (demonstrated with the blue line)
on UTMs, we only able to allow access on Layer 3 and 4; so, how can we handle this kind of situation in Kubernetes architecture? can we somehow change the abstraction of IPs from Nodes to the service or namespace level?
keep in mind, we can have multiple applications on our k8s cluster and each app must have it's own backend accesses and shouldn't have access to any other backed services(outside of the cluster) that are just needed for some other specific applications.
I couldn't find any solution to handle this in an standard and straightforward way and I was wondering that Kubernetes clusters are implemented on big enterprises, how can they handle managing this kind of access problems with the security teams?
thank you for your time!
