Minikube volume write permissions?

Question

The big picture is: I'm trying to install WordPress with plugins in Kubernetes, for development in Minikube.

I want to use the official wp-cli Docker image to install the plugins. I am trying to use a write-enabled persistence volume. In Minikube, I turn on the mount to minikube cluster with command:

minikube mount ./src/plugins:/data/plugins

Now, the PV definition looks like this:

---
apiVersion: v1
kind: PersistentVolume
metadata:
    name: wordpress-install-plugins-pv
    labels:
        app: wordpress
        env: dev
spec:
    capacity:
        storage: 5Gi
    storageClassName: ""
    volumeMode: Filesystem
    accessModes:
        - ReadWriteOnce
    hostPath:
        path: /data/plugins

The PVC looks like this:

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
    name: wordpress-install-plugins-pvc
    labels:
        app: wordpress
spec:
    accessModes:
        - ReadWriteOnce    
    resources:
        requests:
            storage: 5Gi
    storageClassName: ""
    volumeName: wordpress-install-plugins-pv

Both the creation and the binding are succesful. The Job definition for plugin installation looks like this:

---
apiVersion: batch/v1
kind: Job
metadata:
    name: install-plugins
    labels:
        env: dev
        app: wordpress
spec:
    template:
        spec:
            securityContext:
              fsGroup: 82 # www-data
            volumes:
                - name: plugins-volume
                  persistentVolumeClaim:
                       claimName: wordpress-install-plugins-pvc                
                - name: config-volume
                  configMap:
                       name: wordpress-plugins
            containers:
            - name: wpcli
              image: wordpress:cli
              volumeMounts:
              - mountPath: "/configmap"
                name: config-volume
              - mountPath: "/var/www/html/wp-content/plugins"
                name: plugins-volume
              command: ["sh", "-c", "id; \
                                     touch /var/www/html/wp-content/plugins/test; \
                                     ls -al /var/www/html/wp-content; \
                                     wp core download --skip-content --force && \
                                     wp config create --dbhost=mysql \
                                                      --dbname=$MYSQL_DATABASE \
                                                      --dbuser=$MYSQL_USER \
                                                      --dbpass=$MYSQL_PASSWORD && \
                                     cat /configmap/wp-plugins.txt | xargs -I % wp plugin install % --activate" ]
              env:
                  - name: MYSQL_USER
                    valueFrom:
                        secretKeyRef:
                            name: mysql-secrets
                            key: username
                  - name: MYSQL_PASSWORD
                    valueFrom:
                        secretKeyRef:
                            name: mysql-secrets
                            key: password
                  - name: MYSQL_DATABASE
                    valueFrom:
                        secretKeyRef:
                            name: mysql-secrets
                            key: dbname
            restartPolicy: Never
    backoffLimit: 3

Again, the creation looks fine and all the steps look fine. The problem I have is that apparently the permissions to the mounted volume do not allow the current user to write to the folder. Here's the log contents:

uid=82(www-data) gid=82(www-data) groups=82(www-data)
touch: /var/www/html/wp-content/plugins/test: Permission denied
total 9
drwxr-xr-x    3 root     root          4096 Mar  1 20:15 .
drwxrwxrwx    3 www-data www-data      4096 Mar  1 20:15 ..
drwxr-xr-x    1 1000     1000            64 Mar  1 17:15 plugins
Downloading WordPress 5.3.2 (en_US)...
md5 hash verified: 380d41ad22c97bd4fc08b19a4eb97403
Success: WordPress downloaded.
Success: Generated 'wp-config.php' file.
Installing WooCommerce (3.9.2)
Downloading installation package from https://downloads.wordpress.org/plugin/woocommerce.3.9.2.zip...
Unpacking the package...
Warning: Could not create directory.
Warning: The 'woocommerce' plugin could not be found.
Error: No plugins installed.

Am I doing something wrong? I tried different minikube mount options, but nothing really helped! Did anyone run into this issue with minikube?

Answer 1

This is a long-term issue that prevents a non-root user to write to a container when mounting a hostPath PersistentVolume in Minikube.

There are two common workarounds:

1. Simply use the root user.

2. Configure a Security Context for a Pod or Container using runAsUser, runAsGroup and fsGroup. You can find a detailed info with an example in the link provided.

Please let me know if that helped.

Answer 2

I looked deeper into the way the volume mount works in minikube, and I think I came up with solution.

TL;DR

minikube mount ./src/plugins:/data/mnt/plugins --uid 82 --gid 82

Explanataion

There are to mounting moments:

• minikube mounting the directory with minikube mount
• volumne being mounted in Kubernetes

minikube mount sets up the directory in the VM with the UID and GID provided as parameters, with the default being docker user and group.

When the volume is being mounted in the Pod as a directory, it gets mounted with the exact same UID and GID as the host one! You can see this in my question:

drwxr-xr-x    1 1000     1000            64 Mar  1 17:15 plugins

UID=1000 and GID=1000 refer to the docker UID and GID in the minikube host. Which gave me an idea, that I should try mounting with the UID and GID of the user in the Pod.

82 is the id of both the user and the group www-data in the wordpress:cli Docker image, and it works!

One last think worth mentioning: the volume is mounted as a subdirectory in the Pod (wp-content in my case). It turned out that wp-cli actually needs access to that directory as well to create temporary folder. What I ended up doing is adding an emptyDir volume, like this:

volumes
  - name: content
    emptyDir: {}

I hope it help anybody! For what it's worth, my version of minikube is 1.7.3, running on OS X with VirtualBox driver.

Answer 3

Unfortunately, for Minikube today, 2 (Configure a Security Context for a Pod or Container using runAsUser, runAsGroup and fsGroup. You can find a detailed info with an example in the link provided.) doesn't seem to be a viable option, because the HostPast provisioner, which is used under the hood, doesn't honor Security Context. There seems to be a newer HostPath Provisioner, which preemptively sets new mounts to 777, but the one that came with my 1.25 MiniKube is still returning these paths as 755.

Answer 4

I am deploying a daemonset which mounts the hostpath-provisioner directory and sets all subdirs to 777 every second. Obviously not great, but I prefer this workaround since it is self-contained to the minikube cluster (no mounting shennanigans), and it is not as intrusive as messing with the provisioner addons (which may or may not work anyways).

apiVersion: v1
kind: Namespace
metadata:
  name: minikube-pv-hack
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: minikube-pv-hack
  namespace: minikube-pv-hack
spec:
  selector:
    matchLabels:
      name: minikube-pv-hack
  template:
    metadata:
      labels:
        name: minikube-pv-hack
    spec:
      terminationGracePeriodSeconds: 0
      containers:
      - name: minikube-pv-hack
        image: registry.access.redhat.com/ubi8:latest
        command:
        - bash
        - -c
        - |
          while : ; do
            chmod 777 /target/*
            sleep 1
          done
        volumeMounts:
        - name: host-vol
          mountPath: /target
      volumes:
      - name: host-vol
        hostPath:
          path: /tmp/hostpath-provisioner/default