What are AccessModes in Kubernetes Volumes

Question

I am trying to understand the Access Modes of Kubernetes PersistentVolumes.

As per the Kubernetes docs, the access modes are:

ReadWriteOnce -- the volume can be mounted as read-write by a single node
ReadOnlyMany -- the volume can be mounted read-only by many nodes
ReadWriteMany -- the volume can be mounted as read-write by many nodes

Volume Plugin HostPath supports ReadWriteOnce

I have a K8s cluster of 1 controlplane and 1 worker1 node I deployed two Pods one to each node, pv, pvc in the same namespace as per the below config. Both the pods running on different nodes are able to Read-Write to the local path /tmp/test.

As per my understanding, this should not happen. Only one node should be able to ReadWrite and another node should only be able to Read.

Can someone explain what is happening here, if possible please provide me with examples/blogs to see the differences between RWO, RWX, ROX? Most of the blogs, just talk about the PV, PVC and access modes in brief.

apiVersion: v1
kind: Pod
metadata:
  name: pod2
spec:
  nodeName: controlplane
  containers:
  - image: nginx
    name: pod2
    volumeMounts:
      - name: vol
        mountPath: /usr/share/nginx/html
  volumes:
  - name: vol
    persistentVolumeClaim:
      claimName: pvc
---
apiVersion: v1
kind: Pod
metadata:
  name: pod1
spec:
  nodeName: worker1
  containers:
  - image: nginx
    name: pod1
    volumeMounts:
      - name: vol
        mountPath: /usr/share/nginx/html
  volumes:
  - name: vol
    persistentVolumeClaim:
      claimName: pvc
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: /tmp/test
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

Fabrice Jammes · Accepted Answer · 2021-03-15T10:29:50.507

please note that you should not run pods on control plane, so you should replace nodeName: controlplane with nodeName: worker2 in you yaml file. Usually you need to use toleration to run pod on control plane, because it has a taint preventing to run regular pod on it...
I tried your yaml on Google Kubernetes Engine (gke), I had to replace nodeName fields to make it works:

diff example.yaml example-gke.yaml 
6c6
<   nodeName: controlplane
---
>   nodeName: gke-qserv-dev-worker-pool-a64a-01bc5238-3zl9 
23c23
<   nodeName: worker1
---
>   nodeName: gke-qserv-dev-worker-pool-a64a-01bc5238-23t3

And as you can see, k8s refuses to create pod2:

kubectl get pods
NAME                                   READY   STATUS              RESTARTS   AGE
pod1                                   1/1     Running             0          6m27s
pod2                                   0/1     ContainerCreating   0          6m27s

Here is the message I get:

kubectl describe pod pod2 | tail -n 1
  Warning  FailedAttachVolume  7m28s                 attachdetach-controller                                Multi-Attach error for volume "pvc-2fb657ce-4678-4e92-9f64-e4b46b9a3ec7" Volume is already used by pod(s) pod1

So, as you can see you cannot mount a PV on two different pods running on two different nodes. So what you get on your side is not a regular k8s behaviour. It could be a side-effect of your storage class, particularly if you use minikube or kind.

I am aware of tolerations/taints. I untainted my `controlplane` just for testing purpose. BTW I am using `kubeadm` to run this cluster for learning purpose. Thanks for the answer, I guess I have an issue with my cluster itself. I don't have GKE, will try with AKS and update. — John Seen, Mar 15 '21 at 09:31

What are AccessModes in Kubernetes Volumes

1 Answers1