1

I am trying to pass user credentials via Kubernetes secret to a mounted, password protected directory inside a Kubernetes Pod. The NFS folder /mount/protected has user access restrictions, i.e. only certain users can access this folder.

This is my Pod configuration:

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  volumes:
  - name: my-volume
    hostPath:
      path: /mount/protected
      type: Directory
    secret:
      secretName: my-secret
  containers:
  - name: my-container
    image: <...>
    command: ["/bin/sh"]
    args: ["-c", "python /my-volume/test.py"]
    volumeMounts:
    - name: my-volume
      mountPath: /my-volume

When applying it, I get the following error:

The Pod "my-pod" is invalid:
* spec.volumes[0].secret: Forbidden: may not specify more than 1 volume type
* spec.containers[0].volumeMounts[0].name: Not found: "my-volume"

I created my-secret according to the following guide:
https://kubernetes.io/docs/tasks/inject-data-application/distribute-credentials-secure/#create-a-secret
So basically:

apiVersion: v1
kind: Secret
metadata:
  name: my-secret
data:
  username: bXktYXBw
  password: PHJlZGFjdGVkPg==

But when I mount the folder /mount/protected with:

spec:
  volumes:
  - name: my-volume
    hostPath:
      path: /mount/protected
      type: Directory

I get a permission denied error python: can't open file '/my-volume/test.py': [Errno 13] Permission denied when running a Pod that mounts this volume path.

My question is how can I tell my Pod that it should use specific user credentials to gain access to this mounted folder?

Jonas
  • 133
  • 1
  • 2
  • 7

2 Answers2

2

You're trying to tell Kubernetes that my-volume should get its content from both a host path and a Secret, and it can only have one of those.

You don't need to manually specify a host path. Kubernetes will figure out someplace appropriate to put the Secret content and it will still be visible on the mountPath you specify within the container. (Specifying hostPath: at all is usually wrong, unless you can guarantee that the path will exist with the content you expect on every node in the cluster.)

So change:

volumes:
- name: my-volume
  secret:
    secretName: my-secret
  # but no hostPath
David Maze
  • 130,717
  • 29
  • 175
  • 215
  • I think I didn't state my problem clearly, so I updated my question. I want to mount an access restricted NFS folder inside a Pod and only certain users have access to this folder. The question is how can the users pass their user credentials such that they get access to this folder (and no Permission denied error). – Jonas Nov 11 '19 at 12:56
  • What’s a “user”? (A serious question: Docker containers and Kubernetes pods generally run as some single user identity and don’t really have a traditional notion of “user” inside the container space, and they’re not generally connected to any sort of external identity scheme; usually the pod as a whole would have access to NFS with appropriate credentials but there’s not a “depending on who you are” concept at all.) – David Maze Nov 11 '19 at 14:02
  • Each "user" (ubuntu user that logs on the control VM that has Kubernetes installed) has a separate Kubernetes namespace and can only run a Pod under this namespace. We use Kubernetes as a scheduler for deep learning job scheduling. The NFS is used to store training data. Since some users do not want others to be able to see their potentially confidential training data, we need access restricted folders for some users and a way that a Pod can see inside them when somehow passing the appropriate credentials. I hope this is possible. – Jonas Nov 11 '19 at 14:52
  • ...so you’re expecting the Secret to contain the NFS credentials, which need to get passed into the volume mount; but once it’s mounted, you’re okay with the pod being able to access the volume. The person doing the deployment needs to have the credentials. – David Maze Nov 11 '19 at 14:59
  • Yes. Each user could create their own secret and pass this secret to the Pod that is deployed. – Jonas Nov 11 '19 at 15:30
  • Is this possible? – Jonas Nov 12 '19 at 14:07
2

I eventually figured out how to pass user credentials to a mounted directory within a Pod by using CIFS Flexvolume Plugin for Kubernetes (https://github.com/fstab/cifs). With this Plugin, every user can pass her/his credentials to the Pod. The user only needs to create a Kubernetes secret (cifs-secret), storing the username/password and use this secret for the mount within the Pod. The volume is then mounted as follows:

  (...)
  volumes:
  - name: test
    flexVolume:
      driver: "fstab/cifs"
      fsType: "cifs"
      secretRef:
        name: "cifs-secret"
      options:
        networkPath: "//server/share"
        mountOptions: "dir_mode=0755,file_mode=0644,noperm"
Jonas
  • 133
  • 1
  • 2
  • 7