18

I am trying to create Kubernetes cluster using three VMs(Master – 10.x.x.4, Node1 – 10.x.x.150, Node2 – 10.x.x.160).

I was able to create the guestbook application successfully following this link: http://kubernetes.io/v1.0/examples/guestbook/. Only one change I made to frontend-service.yaml: to use NodePort. I can access the frontend service using nodes IP and port number(10.x.x.150:30724 or 10.x.x.160:30724). So everything is working as expected but I am not able to access the frontend service using ClusterIP address(in my case 10.x.x.79).

My understanding of NodePort is that the service can be accessed through cluster IP and also on a port on each node of the cluster. How can I access the service through ClusterIP so that I don’t have to access the each node? Am I missing something here?

service and pod details

$sudo kubectl describe service frontend

Name:                   frontend
Namespace:              default
Labels:                 name=frontend
Selector:               name=frontend
Type:                   NodePort
IP:                     10.x.x.79
Port:                   <unnamed>       80/TCP
NodePort:               <unnamed>       30724/TCP
Endpoints:              172.x.x.13:80,172.x.x.14:80,172.x.x.11:80
Session Affinity:       None

No events.

$sudo kubectl describe pod frontend-2b5us

Name:                           frontend-2b5us
Namespace:                      default
Image(s):                       gcr.io/google_samples/gb-frontend:v3
Node:                           10.x.x.150/10.x.x.150
Labels:                         name=frontend
Status:                         Running
Reason:
Message:
IP:                             172.x.x.11
Replication Controllers:        frontend (3/3 replicas created)
Containers:
  php-redis:
    Image:              gcr.io/google_samples/gb-frontend:v3
    State:              Running
      Started:          Fri, 30 Oct 2015 04:00:40 -0500
    Ready:              True
    Restart Count:      0

I tried to search but would not find any solution for my exact problem but I did find similar problem that looks like for GCE.

Why can't I access my Kubernetes service via its IP?

user7610
  • 25,267
  • 15
  • 124
  • 150
Ethiraj Krishna
  • 203
  • 1
  • 2
  • 8

2 Answers2

17

You do not have ClusterIP service. You do have a NodePort service. To access it, you connect to the NodePort on any of your nodes in the cluster, as you've already discovered. You do get load-balancing here. Even though you connect to a cluster node, the pod you get does not necessarily run on that particular node.

Read the relevant section in the documentation at https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services---service-types to learn about additional service types. You probably do not want NodePort on GCP.

Talking about ClusterIP. To access a ClusterIP service for debugging purposes, you can run kubectl port-forward. You will not actually access the service, but you will directly connect to one of the pods.

For example

kubectl port-forward frontend-2b5us 80 8080

Now connect to localhost:8080

More sophisticated command, which discovers the port on its own, given namespace -n weave and a selector. Taken from https://www.weave.works/docs/scope/latest/installing/

kubectl port-forward -n weave \
    "$(kubectl get -n weave pod \
         --selector=weave-scope-component=app \
         -o jsonpath='{.items..metadata.name}')" \
    4040
user7610
  • 25,267
  • 15
  • 124
  • 150
5

From where are you trying to access clusterIP? The clusterIP (by default) only works from within the cluster. It is a virtual IP, not routed.

Tim Hockin
  • 3,567
  • 13
  • 18
  • I am trying to access within cluster first and also how to configure to access the clusterIP oudside of cluster? I am trying everything beyond firwirewall. when I say outside I don't mean(public). – Ethiraj Krishna Oct 30 '15 at 17:35
  • From within the cluster the clusterIP should work. Can you explain what errors you get? You might also want to read http://docs.k8s.io/v1.0/user-guide/debugging-services.html – Tim Hockin Oct 30 '15 at 18:05
  • Thanks Tim, I am just following the debug procedures now and it looks like service, endpoints looks good and also I can access the service using service’s IP within node. I think kube-proxy is the problem, I am attaching the error which I see under both nodes(/var/log/upstart/ kube-proxy.log) but I have no clue what this error means. While I am trying to debug and understand myself(I totally new to Kubernetes), if you have any idea how to debug further let me know. Thanks again..Please see the error msg below. – Ethiraj Krishna Oct 30 '15 at 19:46
  • var/log/upstart/ kube-proxy.log W1030 09:43:24.290044 17714 api.go:153] Got error status on WatchServices channel: &{TypeMeta:{Kind: APIVersion:} ListMeta:{SelfLink: ResourceVersion:} Status:Failure Message:401: The event in requested index is outdated and cleared (the requested history has been cleared [15572/15326]) [16571] Reason: Details: Code:0} E1030 09:58:27.504744 17714 proxysocket.go:99] Dial failed: dial tcp 172.16.82.11:80: i/o timeout – Ethiraj Krishna Oct 30 '15 at 19:48
  • You said "I can access the service using service’s IP within node". That means kube-proxy is working. Does that mean that you can not access the service's IP from a pod? What part is failing and what happens when it fails (error message?) – Tim Hockin Oct 30 '15 at 20:16
  • I do $curl 10.x.x.79:80 in the nodes and I get response back. but when do outside of nodes or in the master I get below error :curl: (7) Failed to connect to 10.x.x.79 port 80: Connection refused. Now I am thinking its working within cluster as you mentioned? Thanks for the link that helped me to test it. Now I am back to my orginal issue, which I need to make this service available outside the cluster. – Ethiraj Krishna Oct 30 '15 at 20:41
  • Right. The clusterIP only works from within the cluster. To expose it you need to either use a node port or a load balancer. http://docs.k8s.io/v1.0/user-guide/services.html#external-services – Tim Hockin Oct 30 '15 at 22:47
  • Thanks for the reply Tim, I am using type = NodePort in my frontend-service.yaml as i mentioned eariler i can access my application using Node's IP and Node port. I trying to figure out way to access one service endpoint externaly so that I don't have to handle the loadbalacing. It seems like type =LoadBalancer might support that but most of the document says its works on cloud providers but not sure how I can use it in my environment. Again this might be my lack of understanding and interpreting the documentation. – Ethiraj Krishna Nov 02 '15 at 15:39
  • LoadBalancer type only works on clouds that support LoadBalancer provisioning. You can set up your own load balancer, eg https://github.com/kubernetes/contrib/tree/master/service-loadbalancer – Tim Hockin Nov 02 '15 at 16:01
  • Thanks, I will try that. – Ethiraj Krishna Nov 02 '15 at 17:46
  • Hi @EthirajKrishna, using a load balancer actually is not always the best solution, especially when youare exposing a database, cloud providers like aws use resolved public dns, in case of a high traffic system, you will suffer from network delay between your app & your database – Nourdine Alouane Mar 03 '17 at 20:38
  • @TimHockin is there a way (documented?) to access ClusterIP services in your browser? Setup a VPN to get an IP from inside your cluster or something like that? – DenCowboy Jan 21 '18 at 16:25
  • It's about routing. The ClusterIP VIPs are not real IPs. If you have a static routing table you can manipulate, you can add a route for the VIP range to your Node(s) and they will answer the VIP. – Tim Hockin Jan 21 '18 at 16:55