What is the difference between subPath and mountPath in Kubernetes

Question

I am trying to add files in volumeMounts to .dockerignore and trying to understand the difference between subPath and mountPath. Reading official documentation isn't clear to me.

I should add from what I read mountPath is the directory in the pod where volumes will be mounted.

from official documentation: "subPath The volumeMounts.subPath property specifies a sub-path inside the referenced volume instead of its root." https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath (this part isn't clear)

- mountPath: /root/test.pem
            name: test-private-key
            subPath: test.testing.com.key

In this example should I include both test.pem and test.testing.com.key to dockerignore?

Answer 1

mountPath shows where the referenced volume should be mounted in the container. For instance, if you mount a volume to mountPath: /a/b/c, the volume will be available to the container under the directory /a/b/c.

Mounting a volume will make all of the volume available under mountPath. If you need to mount only part of the volume, such as a single file in a volume, you use subPath to specify the part that must be mounted. For instance, mountPath: /a/b/c, subPath: d will make whatever d is in the mounted volume under directory /a/b/c

Answer 2

The difference between mountPath & subPath is that subPath is an addition to mountPath and it exists to solve a problem.

Look at my comments inside the example Pod manifest, I explain the problem and how subPath solves it.

To further understand the difference look at the "under the hood" section to see how kubernetes treats these two properties.

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  volumes:
  - name: vol1
    emptyDir: {}
    
  containers:
  - image: nginx
    name: mypod-container

    volumeMounts:

      # In our example let's say we want to mount "vol1" on path "/a/b/c"
      # So we declare this:

    - name: vol1
      mountPath: /a/b/c

      # But what if we also want to use a child folder "d" ?
      # If we try to use "/a/b/c/d" then we wont have access to /a/b/c
      # because the mountPath /a/b/c is overwritten by mountPath /a/b/c/d
      # So if we try the following mountPath we lose our above declaration:

#   - name: vol1
#     mountPath: /a/b/c/d  # This overwrites the above mount to /a/b/c


      # The solution:
      # Using subPath we enjoy both worlds.
      # (1) We dont overwrite the info in our volume at path /a/b/c .
      # (2) We have a separate path /a/b/c/d that when we can write to 
      #     without affecting the content in path /a/b/c.
      # Here is how we write the correct declaration:

    - name: vol1
      mountPath: /a/b/c/d
      subPath: d

---

Under the hood of mountPath & subPath

Lets look under the hood of kubernetes to see how it manages the mountPath & the subPath properties differently:

1. How kubernetes manages mountPath:

When a mountPath is declared then kubernetes creates a file with the name of the volume in the following path:

/var/lib/kubelet/pods/<pod-id>/volumes/kubernetes.io~empty-dir/<volume name>

So in our above manifest example this is what was created ("vol1" is the volume name): /var/lib/kubelet/pods/301d...a71c/volumes/kubernetes.io~empty-dir/vol1

Now you can see that if we defined the "/a/b/c/d" mountPath we would have triggered a creation of another file "vol1" in same directory thus overwritting the original.

2. How kubernetes manages subPath:

When a subPath is declared then kubernetes creates a file with the SAME VOLUME name Volume but in a different path:

/var/lib/kubelet/pods/<pod-id>/volume-subpaths/<volume name>

So in our above manifest example this is what was created ("vol1" is the volume name): /var/lib/kubelet/pods/3eaa...6d1/volume-subpaths/vol1

Conclusion:

Now you see that the subPath enables us to define an additional volumePath without colliding with the root voulmePath. It does this by creating files with the same volume name but in different directories in kubernetes kubelet.

Answer 3

subPath is used to select a specific Path within the volume from which the container's volume should be mounted. This Defaults to "" (volume's root).

Check this mentioned here. So what that means is that you can still mount a volume on the path mentioned in your mountPath but instead of mounting it from the volume root, you can specify a separate subpath within the volume to be mounted under the volumeMount directory in your container.

To clarify on what this means, i created a simple volume on my minikube node.

docker@minikube:/mnt/data$ ls -lrth
total 8.0K
drwxr-xr-x 2 root root 4.0K Dec 30 16:23 path1
drwxr-xr-x 2 root root 4.0K Dec 30 16:23 path2
docker@minikube:/mnt/data$ 
docker@minikube:/mnt/data$ pwd
/mnt/data

As you can see that in this case, i have a directory and i have created two sub directories inside this volume. Under each of these path1 or path2 folder i have placed a different index file.

docker@minikube:/mnt/data$ pwd
/mnt/data
docker@minikube:/mnt/data$ cat path1/index.html 
This is index file from path1
docker@minikube:/mnt/data$ 
docker@minikube:/mnt/data$ cat path2/index.html 
This is index file from path2
docker@minikube:/mnt/data$ 

Now I created a sample PV using this volume on my minikube node using the sample manifest as below

apiVersion: v1
kind: PersistentVolume
metadata:
  name: task-pv-volume
  labels:
    type: local
spec:
  storageClassName: manual
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"

After this, i created the sample PVC using the below manifest

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: task-pv-claim
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 800Mi

Now if i created a nginx pod and used this PVC under my volume, depending on the subPath config that i use in my pod spec, i will have the volume mounted from that specific subfolder.

i.e. If i used the below manifest for my nginx pod

apiVersion: v1
kind: Pod
metadata:
  name: test
spec:
  containers:
    - name: test
      image: nginx
      volumeMounts:
        # a mount for site-data
        - name: config
          mountPath: /usr/share/nginx/html
          subPath: path1
  volumes:
    - name: config
      persistentVolumeClaim:
        claimName: task-pv-claim

and i do a curl on the POD IP, i get the index.html served from path1.

Gauravs-MBP:K8s alieninvader$ kubectl exec -it mycurlpod -- /bin/sh
/ $ curl 172.17.0.3
This is index file from path1

And if i changed my pod manifest and used the subpath as path2, so that the new manifest becomes this

apiVersion: v1
kind: Pod
metadata:
  name: test
spec:
  containers:
    - name: test
      image: nginx
      volumeMounts:
        # a mount for site-data
        - name: config
          mountPath: /usr/share/nginx/html
          subPath: path2
  volumes:
    - name: config
      persistentVolumeClaim:
        claimName: task-pv-claim

Then as expected the curl to nginx pod would produce the below output serving the file from path2.

Gauravs-MBP:K8s alieninvader$ kubectl exec -it mycurlpod -- /bin/sh

/ $ curl 172.17.0.3
This is index file from path2
/ $