I'm trying to set up CMEK in my cluster as per the details mentioned here: https://cloud.google.com/kubernetes-engine/docs/how-to/dynamic-provisioning-cmek#dynamically_provision_an_encrypted
I have deployed the Compute Engine Persistent Disk CSI Driver to my cluster as per the steps mentioned in: https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/blob/master/docs/kubernetes/development.md
I have then created the key/key ring and have created the below storage class:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-gce-pd
provisioner: pd.csi.storage.gke.io
parameters:
type: pd-standard
disk-encryption-kms-key: "projects/xx/locations/us-central1/keyRings/xx/cryptoKeys/xx
Below is the YAML for the PVC:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: encrypt-pvc
spec:
accessModes:
- ReadWriteOnce
storageClassName: csi-gce-pd
resources:
requests:
storage: 5Gi
However, when i apply the PVC YAML, it fails with the below error and PVC status will be at pending:
Name: encrypted-pvc
Namespace: gce-pd-csi-driver
StorageClass: csi-gce-pd
Status: Pending
Volume:
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{"volume.beta.kubernetes.io/storage-class":"csi-gce-pd"},"nam...
volume.beta.kubernetes.io/storage-class: csi-gce-pd
volume.beta.kubernetes.io/storage-provisioner: pd.csi.storage.gke.io
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Filesystem
Mounted By: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Provisioning 4s (x3 over 15s) pd.csi.storage.gke.io_csi-gce-pd-controller-0_5c51fedd-8092-4c71-aca9-5a13b566bb8a External provisioner is provisioning volume for claim "gce-pd-csi-driver/encrypted-pvc"
Normal ExternalProvisioning 2s (x2 over 15s) persistentvolume-controller waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by system administrator
Warning ProvisioningFailed 0s (x3 over 11s) pd.csi.storage.gke.io_csi-gce-pd-controller-0_5c51fedd-8092-4c71-aca9-5a13b566bb8a failed to provision volume with StorageClass "csi-gce-pd": rpc error: code = Internal desc = CreateVolume failed to create single zonal disk "pvc-1524bf19-f6f1-11e9-a706-4201ac100007": failed to insert zonal disk: unkown Insert disk error: googleapi: Error 400: Invalid resource usage: 'Cloud KMS error when using key projects/acn-devopsgcp/locations/us-central1/keyRings/testkeyring1/cryptoKeys/testkey1: Permission 'cloudkms.cryptoKeyVersions.useToEncrypt' denied on resource 'projects/acn-devopsgcp/locations/us-central1/keyRings/testkeyring1/cryptoKeys/testkey1' (or it may not exist).'., invalidResourceUsage
I have given the below roles to the service account and the KMS key resource identifier is also correct. Cloud KMS CryptoKey Encrypter/Decrypter Cloud KMS CryptoKey Encrypter Cloud KMS CryptoKey Decrypter
Kubectl version :
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T19:18:23Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.7-gke.10", GitCommit:"8cea5f8ae165065f0d35e5de5dfa2f73617f02d1", GitTreeState:"clean", BuildDate:"2019-10-05T00:08:10Z", GoVersion:"go1.12.9b4", Compiler:"gc", Platform:"linux/amd64"}
