What Persistent Volume storage solutions I can use when I run my Kubernetes cluster on GCP?

Question

I have deployed my Kubernetes cluster on GCP Compute Engines and having 3 Master Nodes and 3 Worker Nodes (It's not a GKE Cluster). Can anybody suggest me what storage options I can use for my cluster? If I create a virtual disk on GCP, can I use that disk as a persistent storage?

Answer 1

You can use GCE Persistent Disk Storage Class.

Here is how you create the storage class:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ssd
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-ssd

Then you do the following to create the PV & PVC and to attach to your pod.

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: gce-claim
spec:
 accessModes:
   - ReadWriteOnce
 storageClassName: ssd
 resources:
   requests:
     storage: 10Gi
---
apiVersion: v1
kind: Pod
metadata:
 name: webserver-pd
spec:
 containers:
 - image: httpd
   name: webserver
   volumeMounts:
   - mountPath: /data
     name: dynamic-volume
 volumes:
 - name: dynamic-volume
   persistentVolumeClaim:
     claimName: gce-claim

Example taken from this blog post

Answer 2

There are two types of provisioning Persistent Volumes: Static Provisioning and Dynamic Provisioning.
I will briefly describe each of these types.

Static Provisioning

Using this approach you need to create Disk, PersistentVolume and PersistentVolumeClaim manually. I've create simple example for you to illustrate how it works.

First I created disk, on GCP we can use gcloud command:

$ gcloud compute disks create --size 10GB --region europe-west3-c test-disk
NAME       ZONE            SIZE_GB  TYPE         STATUS
test-disk  europe-west3-c  10       pd-standard  READY 

Next I created PV and PVC using this manifest files:

--- 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-test
spec:
  accessModes:
    - ReadWriteOnce  
  capacity:
    storage: 10Gi
  gcePersistentDisk:
    pdName: test-disk # This GCE PD must already exist.
    fsType: ext4
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: claim-test
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  

After applaying this manifest files, we can check status of PV and PVC:

root@km:~# kubectl get pv,pvc
NAME                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                STORAGECLASS   REASON   AGE
persistentvolume/pv-test   10Gi       RWO            Retain           Bound    default/claim-test                           12m

NAME                               STATUS   VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/claim-test   Bound    pv-test   10Gi       RWO                           12m

Finally I used above claim as volume:

apiVersion: v1
kind: Pod
metadata:
  name: web
spec:
  containers:
    - name: web
      image: nginx
      volumeMounts:
      - mountPath: "/usr/share/nginx"
        name: vol-test
  volumes:
    - name: vol-test
      persistentVolumeClaim:
        claimName: claim-test

We can inspect created Pod to check if it works as expected:

root@km:~# kubectl exec -it web -- bash
root@web:/# df -h
Filesystem      Size  Used Avail Use% Mounted on
...
/dev/sdb        9.8G   37M  9.8G   1% /usr/share/nginx
...

Dynamic Provisioning

In this case volume is provisioned automatically when application requires it. First you need to create StorageClass object to define a provisioner such e.g. kubernetes.io/gce-pd.
We don't need to create PersistenVolume anymore, it's created automatically by StorageClass for us.
I've also created simple example for you to illustrate how it works.

First I created StorageClass as default storage class:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
  name: standard
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-standard
  fstype: ext4

And then PVC (the same as in the previous example) - but in this case PV was created automatically:

root@km:~# kubectl get pv,pvc
NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                STORAGECLASS   REASON   AGE
persistentvolume/pvc-8dcd69f1-7081-45a7-8424-cc02e61a4976   10Gi       RWO            Delete           Bound    default/claim-test   standard                3m10s

NAME                               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/claim-test   Bound    pvc-8dcd69f1-7081-45a7-8424-cc02e61a4976   10Gi       RWO            standard       3m12s

In more advanced cases it may be useful to create multiple StorageClasses with differnet persistent disks types.