3

I'm teaching myself Kubernetes with a 5 Rpi cluster, and I'm a bit confused by the way Kubernetes treats Persistent Volumes with respect to Pod Scheduling.

I have 4 worker nodes using ext4 formatted 64GB micro SD cards. It's not going to give GCP or AWS a run for their money, but it's a side project.

Let's say I create a Persistent volume Claim requesting 10GB of storage on worker1, and I deploy a service which relies on this PVC, is that service then forced to be scheduled on worker1?

Should I be looking into distributed file systems like Ceph or Hdfs so that Pods aren't restricted to being scheduled on a particular node?

Sorry if this seems like a stupid question, I'm self taught and still trying to figure this stuff out! (Feel free to improve my tl;dr doc for kubernetes with a pull req)

Nick Saccente
  • 150
  • 1
  • 9
  • If you have shared storage (ceph, nfs, etc), pods are not tied to a particular node because they can access their storage from all nodes. If your PV only exists on a single node, then of course your pod needs to run there to access it. – larsks Sep 18 '20 at 02:32

2 Answers2

2

just some examples, as already mentioned it depends on your storage system, as i see you use the local storage option

Local Storage: yes the pod needs to be run on the same machine where the pv is located (your case)

ISCSI/Trident San: no, the node will mount the iscsi block device where the pod will be scheduled (as mentioned already volume binding mode is an important keyword, its possible you need to set this to 'WaitForFirstConsumer')

NFS/Trident Nas: no, its nfs, mountable from everywhere if you can access and auth against it

VMWare VMDK's: no, same as iscsi, the node which gets the pod scheduled mounts the vmdk from the datastore

ceph/rook.io: no, you get 3 options for storage, file, block an object storage, every type is distributed so you can schedule a pod on every node. also ceph is the ideal system for carrying a distributed software defined storage on commodity hardware, what i can recommend is https://rook.io/ basically an opensource ceph on 'container-steroids'

Elytscha Smith
  • 1,351
  • 1
  • 7
  • 11
1

Let's say I create a Persistent volume Claim requesting 10GB of storage on worker1, and I deploy a service which relies on this PVC, is that service then forced to be scheduled on worker1?

This is a good question. How this works depends on your storage system. The StorageClass defined for your Persistent Volume Claim contains information about Volume Binding Mode. It is common to use dynamic provisioning volumes, so that the volume is first allocated when a user/consumer/Pod is scheduled. And typically this volume does not exist on the local Node but remote in the same data center. Kubernetes also has support for Local Persistent Volumes that are physical volumes located on the same Node, but they are typically more expensive and used when you need high disk performance and volume.

Jonas
  • 121,568
  • 97
  • 310
  • 388