Microk8s reaches internet but not internal network

Question

I've just installed Ubuntu 22.04 on a vmware virtual server and started using microk8s. The server is part of a local network in which there are some servers, including microsoft AD and IIS servers that handle the network.

I've docker installed on the ubuntu system and can run all the containers of the web app with no problem via docker. In particular, I have a service (a container) that connects to the windows AD server of the local network to authenticate users of the web app. On the host, it works with no problem, can reach the AD server and also other servers in the network and do all the necessary operations.

On the other hand, when run on kubernetes via microk8s, all the services work, they are all reachable from the local network, while at the same time the containers can reach the external network (outside our local network, e.g. www.google.com). Only the internal network seems to be unreachable, for which I always get a timeout error.

What I tried (but did not work)

External service [https://kubernetes.io/docs/concepts/services-networking/service/#services-without-selectors][1]
Check the dns resolution on the host that gets copied into the container

Note

I'm not sure what kind of commands shall be run in order to provide the most useful information about the configuration, so I'll be iterating over this question, extending it with logs and other meaningful information.

Thanks

Edit 11/10/2022

I've enable the following addons

microk8s is running
high-availability: no
  datastore master nodes: 127.0.0.1:19001
  datastore standby nodes: none
addons:
  enabled:
    dns                  # (core) CoreDNS
    ha-cluster           # (core) Configure high availability on the current node
    helm                 # (core) Helm - the package manager for Kubernetes
    helm3                # (core) Helm 3 - the package manager for Kubernetes
    ingress              # (core) Ingress controller for external access
    metallb              # (core) Loadbalancer for your Kubernetes cluster

Another strange thing, is that the containers can access the postgres database on the host via the host's ip address (10.1.1.xxx)

Edit 2 12/10/2022

Here's the ingress yaml file

apiVersion: v1
kind: Service
metadata:
  name: ingress
  namespace: ingress
spec:
  selector:
    name: nginx-ingress-microk8s
  type: LoadBalancer
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
    - name: https
      protocol: TCP
      port: 443
      targetPort: 443
---
#
# Ingress
#
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: main-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - http:
      paths:
      - path: /api/erp(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: erp-service
            port:
              number: 8000
      - path: /api/auth(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: auth-service
            port:
              number: 8000
      - path: /()(.*)
        pathType: Prefix
        backend:
          service:
            name: ui-service
            port:
              number: 3000

I can access the UI and by using the host's ip and /api/auth, I can access the online documentation of swagger/openapi. [1]: https://kubernetes.io/docs/concepts/services-networking/service/#services-without-selectors

It could be ingress, enable ingress addon, microk8s enable ingress and enable the ports(tcp ports) on the config map that you want to pass. @lsabi — DBSand, Oct 11 '22 at 21:08
I would suggest running a simple hello word app to rule any other issues on ur cluster, to check if ingress traffic is being received https://www.jeffgeerling.com/blog/2022/quick-hello-world-http-deployment-testing-k3s-and-traefik — DBSand, Oct 12 '22 at 19:56
Also, when you run docker do you run a ingress container which configures the ports to receive traffic? Microk8s ingres - > routes to your ingress controller in the cluster which will have the ingress rules. — DBSand, Oct 12 '22 at 19:58
@DBSand yes, traffic is being received, as I can access swagger/openapi on different services (included the authentication one). 2) I can navigate to the services, but in case I edited my question with the configuration file — lsabi, Oct 12 '22 at 20:26
Could you check in Ingress namespace, the nginx-ingress-microk8s-daemonset, check if the ports have been opened, you can open it, this is opened for microk8s ingress which will allow traffic to your controller, by mentioning the ports: containerPort: 80 hostport: 80 name: http protocol: TCP, similarly for 443. — DBSand, Oct 13 '22 at 16:04
Thanks @DBSand for the support, but it seems I'm not the only one struggling with this problem. Could be a bug or some strange configuration due to the virtualization. Anyways, I found a workaround for the moment, so I'll use it — lsabi, Oct 17 '22 at 21:10

score 0 · Accepted Answer · answered Oct 17 '22 at 21:09

To this day I haven't managed to find any solution but to circumvent the request and use a "proxy" endpoint as suggested in

Accessing an external InfluxDb Database from a microk8s pod using selectorless service and manual endpoint?

Basically, it creates a service with that can be accessed by the cluster and an endpoint that points to the external resource.

Actual source config taken from the answer

kind: Service
apiVersion: v1
metadata:
  name: influxdb-service-lb
  #namespace: ingress
spec:
  type: LoadBalancer
  loadBalancerIP: 10.1.2.61
#  selector:
#    app: grafana
  ports:
  - name: http
    protocol: TCP
    port: 8086
    targetPort: 8086
---
apiVersion: v1
kind: Endpoints
metadata:
  name: influxdb-service-lb
subsets:
  - addresses:
      - ip: 10.1.2.220
    ports:
      - name: influx
        protocol: TCP
        port: 8086

If I'll manage to find a solution, I'll update this answer