K8s - Cannot access service/clusterIP from another node

Question

I am trying to access(from worker node) a pod(on worker node) via a service/ClusterIP using curl http://cluster_ip:port_no but it isn't working.

Here's some info on service

masternode@Master:/localdocker$ kubectl describe svc registry
Name:              registry
Namespace:         default
Labels:            io.kompose.service=registry
Annotations:       kompose.cmd: kompose convert -f docker-compose.yaml -o localregistry.yaml
                   kompose.version: 1.1.0 (36652f6)
Selector:          io.kompose.service=registry
Type:              ClusterIP
IP:                10.100.126.230
Port:              5000  5000/TCP
TargetPort:        5000/TCP
Endpoints:         192.168.171.74:5000
Session Affinity:  None
Events:            <none>```

here's some info on pod

masternode@Master:/localdocker$ kubectl describe pod registry-7ccd695dc7-69cx4
Name:         registry-7ccd695dc7-69cx4
Namespace:    default
Priority:     0
Node:         worker/10.0.1.5
Start Time:   Sun, 19 Jul 2020 06:09:14 +0000
Labels:       io.kompose.service=registry
              pod-template-hash=7ccd695dc7
Annotations:  cni.projectcalico.org/podIP: 192.168.171.74/32
              cni.projectcalico.org/podIPs: 192.168.171.74/32
Status:       Running
IP:           192.168.171.74
IPs:
  IP:           192.168.171.74
Controlled By:  ReplicaSet/registry-7ccd695dc7
Containers:
  registry:
    Container ID:   docker://ca372f12ef7a1a3cb23e7d6c58337f47848f91212f6c75af6bfd04bc48ea2f27
    Image:          registry:2
    Image ID:       docker-pullable://registry@sha256:8be26f81ffea54106bae012c6f349df70f4d5e7e2ec01b143c46e2c03b9e551d
    Port:           5000/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Sun, 19 Jul 2020 06:09:24 +0000
    Ready:          True
    Restart Count:  0
    Environment:
      REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY:  /data
    Mounts:
      /data from registry-claim0 (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-fhf5k (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  registry-claim0:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  registry-claim0
    ReadOnly:   false
  default-token-fhf5k:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-fhf5k
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  13m   default-scheduler  Successfully assigned default/registry-7ccd695dc7-69cx4 to worker
  Normal  Pulling    13m   kubelet, worker    Pulling image "registry:2"
  Normal  Pulled     13m   kubelet, worker    Successfully pulled image "registry:2"
  Normal  Created    13m   kubelet, worker    Created container registry
  Normal  Started    13m   kubelet, worker    Started container registry

This is a practice exercise where I was able to do so(in their live environment) without any NodePorts.

Please let me know if any other info is required.

Arghya Sadhu · Answer 1 · 2020-07-19T07:51:49.463

5

This is an expected behavior because ClusterIP type service is only accessible from within the kubernetes cluster i.e from another pod etc.

If you want to access a pod via a service from outside the kubernetes cluster i.e from the nodes itself then use NodePort type service.

Once you expose it via NodePort service you would be able to access it using curl http://<NODE-IP>:<NODE-PORT>

ClusterIP is created on service network of the cluster and nodes are in different network. By creating a NodePort service a Port is opened in each nodes network to forward traffic to ClusterIP. So in essence NodePort uses ClusterIP internally and is an higher level abstraction built on top of ClusterIP.

edited Jul 19 '20 at 07:51

answered Jul 19 '20 at 07:39

Arghya Sadhu

41,002
9
78
107

1

So you mean to say, nodes don't become a part of the cluster? Also, why would I then need a clusterIP/service as pods can communicate with each other without NAT? NodePorts are for exposing pod's port to node's external port for public access. – Aakash Verma Jul 19 '20 at 07:48
I can very well use the IP of a pod to communicate between two pods, but I believe if I am using a service it exposes the pods to be used by other nodes and pods. – Aakash Verma Jul 19 '20 at 07:49
Pod IPs change if the pod is restarted which is pretty common in kubernetes ..Service IPs are stable..also creating a clusterIP service gives a DNS based name which can be used to communicate between pods within the cluster. – Arghya Sadhu Jul 19 '20 at 07:54
I've an exercise where I am supposed to build a local registry on a node and use another node to pull an image out of the registry onto itself. The services have no NodePorts in them :/ – Aakash Verma Jul 19 '20 at 08:15
can you please check the question again? – Aakash Verma Jul 19 '20 at 14:42
It's all docker..I don't see any need of kubernetes there – Arghya Sadhu Jul 19 '20 at 14:45
the commands are docker based but the registry is pod on master ..from which images are being pulled into worker node – Aakash Verma Jul 19 '20 at 14:58
So what is the issue or error again? It's technically not possible to access a pod from outside the cluster using clusterIP..So either the question is wrong or we are misunderstanding it – Arghya Sadhu Jul 19 '20 at 15:06
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/218165/discussion-between-aakash-verma-and-arghya-sadhu). – Aakash Verma Jul 19 '20 at 18:56

K8s - Cannot access service/clusterIP from another node

1 Answers1