0

I am currently in the process of designing a Kubernetes cluster for an existing setup which includes around ten services. Previously, I've developed and tested these services locally with minikube and later on I published them a self-hosted Kubernetes instance on a different physical machine (let's call this the staging system). Some of these deployed services need to access resources on other machines in the same network. This means that my development environment (minikube) also needs access to those external resources. Here is a short "mockup" of one process in this cluster.

+-----------+
| External  |
| Service X |
+-----------+
    ^
    |
+-----------+       +-----------+
| Service A |  <--  | Service B |
+-----------+       +-----------+

Due to some networking changes it is no longer possible for me to access these remote resources, that means that External Service X is no longer reachable from my local machine. I'd know ask if it would be somehow possible to host the Service B on my local machine, but to "connect" it to my remote cluster. Is it somehow possible to use minikube as a simple node for an existing Kubernetes instance? I rely heavily on service discovery with CoreDNS which means I'd like to find an infrastructure-level solution instead of an application-level approach. It would be ideal if the staging system didn't even really know that my Service B is not locally hosted. I am sorry if this is maybe a stupid question but it seems like I am a bit out of knowledge about Kubernetes at the moment.

user13237363
  • 23
  • 1
  • 2
  • **Minikube** is designed for development/testing purposes and you shouldn't rely on it as a part of your production infrastructure. Anyway, you won't be able to make it integral part of other kubernetes cluster. It doesn't work like that. Both clusters may connect with each other if needed as separate independent environments. Do you need your **Minikube** to be accessed from other machines ? Or you need it only to be able to access services deployed on those machines ? It's difficult to give you straightforward answer as we don't know anything about your infrastructure. – mario Apr 06 '20 at 19:39
  • I feel like I already explained the answer to your question but maybe it was not stated clearly. I am not trying to make minikube part of my production/staging cluster! I am trying to find a way to develop&run services locally with them interacting with the production/staging cluster. I want to ONLY run a single service on minikube and have it interact with the other services in the cluster. I need to do this because 1. if I execute them locally they can't access the external service and 2. my dev machine isn't strong enough to casually handle running 10+ Spring Boot microservices in parallel. – user13237363 Apr 07 '20 at 13:40
  • *"Is it somehow possible to use minikube as a simple node for an existing Kubernetes instance?"* - doesn't it suggest an idea of making minikube kind of integral part of another cluster ? That's why I'm asking. You wrote that *"... services need to access resources on other machines **in the same network**."*. If there is network connectivity between them you can utilize so called [Services without selectors](https://kubernetes.io/docs/concepts/services-networking/service/#services-without-selectors) to configure access to an external resources (running on prod k8s) in your local k8s cluster. – mario Apr 07 '20 at 14:18
  • The biggest advantage of such solution is that such `Service` (I'm referring here to `Service` as specific k8s resource, which exposes your workload *within your cluster* or *externally*, not as a *Microservice*) makes specific external resource available internally to apps runnig in `Pods` on your k8s cluster and they can refer to it using its DNS name i.e. `service-name` within the same `namespace` or by FQDN `..svc.cluster.local` from any namespace, cluster-wide. Take also a look at [this](https://stackoverflow.com/a/60096013/11714114) answer. – mario Apr 07 '20 at 14:41
  • Wow, that sounds exactly like what I need. Thank you for the great help! – user13237363 Apr 07 '20 at 22:09
  • You're welcome! In the first moment I couldn't simply figure out if this is what you really need. :) If your external resource is available under a `domain name` you may also consider using [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname), which is a special type of *service without selector*, and in this case you don't have to configure manually an *endpoint* object. – mario Apr 07 '20 at 22:29
  • If it was helpful, let me convert those comments into an answer so it can be accepted and and more easily found by others who encounter similar problem. – mario Apr 07 '20 at 22:34
  • Yeah, great. Move it into an answer and I'll accept it. – user13237363 Apr 08 '20 at 19:04

1 Answers1

0

If you have two separate kubernetes clusters e.g. local instance of Minikube and some other remote kubernetes cluster and you want them to be able to interact with one another, you can use for that purpose so called Services without selectors.

You can treat your remote kubernetes cluster as any other external resource (e.g. database server running on a separate machine/vm) and you can expose it internally to your cluster. Typically we expose something which is running on our k8s cluster externally but in this case we want to expose an external resource internally to our cluster as if it was one more internal resource. From Pods perspective it actually looks like an internal resource, which is available under its local cluster domain, in exactly the same way as internal resources, exposed by simple ClusterIP service.

Let's take a quick look how to achive it. It's very well explained in this section of the official kubernetes docs:

Services most commonly abstract access to Kubernetes Pods, but they can also abstract other kinds of backends. For example:

  • You want to have an external database cluster in production, but in your test environment you use your own databases.
  • You want to point your Service to a Service in a different Namespace or on another cluster.
  • You are migrating a workload to Kubernetes. Whilst evaluating the approach, you run only a proportion of your backends in Kubernetes.

In any of these scenarios you can define a Service without a Pod selector. For example:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376

Because this Service has no selector, the corresponding Endpoint object is not created automatically. You can manually map the Service to the network address and port where it’s running, by adding an Endpoint object manually:

apiVersion: v1
kind: Endpoints
metadata:
  name: my-service
subsets:
  - addresses:
      - ip: 192.0.2.42
    ports:
      - port: 9376

If there is only network connectivity between mentioned dev and staging environments you can make use of Services without selectors. If you need your Minikube cluster to be able to connect to something exposed on 192.0.2.42:9376 by your staging cluster, you just need to use this IP in your Endpoints object definition. Service is defined to listen on por 80 so it will be eventually exposed internally to you Minikube Pods under this standard port.

The biggest advantage of such solution is that such Service makes specific external resource available internally to apps runnig in Pods on your local k8s cluster and they can refer to it using its DNS name i.e. service-name within the same namespace or by FQDN <my-service-name>.<namespace>.svc.cluster.local from any namespace, cluster-wide.

If your external resource is available under a domain name you may also consider using ExternalName, which is a special type of service without selector, and in this case you don't have to configure manually any Endpoints objects. Its definition may look like this:

apiVersion: v1
kind: Service
metadata:
  name: my-service
  namespace: prod
spec:
  type: ExternalName
  externalName: my.database.example.com
mario
  • 9,858
  • 1
  • 26
  • 42