I'm trying to set up CMEK in my cluster as per the details mentioned here: https://cloud.google.com/kubernetes-engine/docs/how-to/dynamic-provisioning-cmek#dynamically_provision_an_encrypted

I have deployed the Compute Engine Persistent Disk CSI Driver to my cluster as per the steps mentioned in: https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/blob/master/docs/kubernetes/development.md

I have then created the key/key ring and have created the below storage class:

apiVersion: storage.k8s.io/v1
kind: StorageClass
  name: csi-gce-pd
provisioner: pd.csi.storage.gke.io
  type: pd-standard
  disk-encryption-kms-key: "projects/xx/locations/us-central1/keyRings/xx/cryptoKeys/xx

Below is the YAML for the PVC:

apiVersion: v1
kind: PersistentVolumeClaim
  name: encrypt-pvc
    - ReadWriteOnce
  storageClassName: csi-gce-pd
      storage: 5Gi

However, when i apply the PVC YAML, it fails with the below error and PVC status will be at pending:

Name:          encrypted-pvc
Namespace:     gce-pd-csi-driver
StorageClass:  csi-gce-pd
Status:        Pending
Labels:        <none>
Annotations:   kubectl.kubernetes.io/last-applied-configuration:
               volume.beta.kubernetes.io/storage-class: csi-gce-pd
               volume.beta.kubernetes.io/storage-provisioner: pd.csi.storage.gke.io
Finalizers:    [kubernetes.io/pvc-protection]
Access Modes:
VolumeMode:    Filesystem
Mounted By:    <none>
  Type     Reason                Age               From                                                                                Message
  ----     ------                ----              ----                                                                                -------
  Normal   Provisioning          4s (x3 over 15s)  pd.csi.storage.gke.io_csi-gce-pd-controller-0_5c51fedd-8092-4c71-aca9-5a13b566bb8a  External provisioner is provisioning volume for claim "gce-pd-csi-driver/encrypted-pvc"
  Normal   ExternalProvisioning  2s (x2 over 15s)  persistentvolume-controller                                                         waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by system administrator
  Warning  ProvisioningFailed    0s (x3 over 11s)  pd.csi.storage.gke.io_csi-gce-pd-controller-0_5c51fedd-8092-4c71-aca9-5a13b566bb8a  failed to provision volume with StorageClass "csi-gce-pd": rpc error: code = Internal desc = CreateVolume failed to create single zonal disk "pvc-1524bf19-f6f1-11e9-a706-4201ac100007": failed to insert zonal disk: unkown Insert disk error: googleapi: Error 400: Invalid resource usage: 'Cloud KMS error when using key projects/acn-devopsgcp/locations/us-central1/keyRings/testkeyring1/cryptoKeys/testkey1: Permission 'cloudkms.cryptoKeyVersions.useToEncrypt' denied on resource 'projects/acn-devopsgcp/locations/us-central1/keyRings/testkeyring1/cryptoKeys/testkey1' (or it may not exist).'., invalidResourceUsage

I have given the below roles to the service account and the KMS key resource identifier is also correct. Cloud KMS CryptoKey Encrypter/Decrypter Cloud KMS CryptoKey Encrypter Cloud KMS CryptoKey Decrypter

Kubectl version :

Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T19:18:23Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.7-gke.10", GitCommit:"8cea5f8ae165065f0d35e5de5dfa2f73617f02d1", GitTreeState:"clean", BuildDate:"2019-10-05T00:08:10Z", GoVersion:"go1.12.9b4", Compiler:"gc", Platform:"linux/amd64"}

I was able to resolve this by providing the roles/cloudkms.cryptoKeyEncrypterDecrypter role to the compute engine default service account as per this link.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.