Bug 1991815 - [KMS] Improve error handling when VAULT_BACKEND is set to "kv" for a backend path using kv-v2
Summary: [KMS] Improve error handling when VAULT_BACKEND is set to "kv" for a backend ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenShift Data Foundation
Classification: Red Hat Storage
Component: documentation
Version: 4.7
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Anjana Suparna Sriram
QA Contact: Elad
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-08-10 05:52 UTC by Rachael
Modified: 2023-08-09 16:43 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-03-02 14:58:20 UTC
Embargoed:


Attachments (Terms of Use)

Description Rachael 2021-08-10 05:52:20 UTC
Description of problem (please be detailed as possible and provide log
snippets):

When VAULT_BACKEND is set to "kv" in the vault config where the backend path is using kv-v2, PV encryption fails. The PVC creation succeeds, however when it is attached to a pod and it tries to create the key in Vault, it fails with the following error:

Warning  FailedMapVolume         8s (x5 over 18s)  kubelet                  MapVolume.SetUpDevice failed for volume "pvc-56732d4c-6203-4f9b-b2a8-0728a8f00135" : rpc error: code = Internal desc = failed 
to encrypt rbd image ocs-storagecluster-cephblockpool/csi-vol-8986b89d-f8f9-11eb-972f-0a580a810211: failed parsing data for get passphrase request for 0001-0011-openshift-storage-0000000000000001-8986b89d-f8f9-11eb-972f-0a580a810211

This could be because the structure of how keys are stored in kv as compared to kv-v2 is different and when use of kv APIs is enforced on a backend path using kv-v2, it fails. The error message does not clearly reflect the cause of the failure and can be improved.

Version of all relevant components (if applicable):
OCS: ocs-operator.v4.7.3-243.ci

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
No, but it is a trivial failure which can be avoided or corrected with proper config values

Is there any workaround available to the best of your knowledge?
Change VAULT_BACKEND to the correct value as that of the backend path

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
2

Can this issue reproducible?
Yes

Can this issue reproduce from the UI?
No

If this is a regression, please provide more details to justify this:
No

Steps to Reproduce:
1. Create a storage class with encryption enabled using a backend path with kv-v2 (https://access.redhat.com/documentation/en-us/red_hat_openshift_container_storage/4.7/html-single/managing_and_allocating_storage_resources/index#creating-a-storage-class-for-persistent-volume-encryption_rhocs)
2. Find out the encryptionKMSID being used for the storageclass
   $ oc get sc <sc_name> -o yaml |grep encryptionKMSID
   encryptionKMSID=4-vault
3. Edit the vault config for the KMSID from the step above in the csi-kms-connection-details configmap
   $ oc edit cm csi-kms-connection-details -n openshift-storage
4. Set the VAULT_BACKEND parameter to "kv" as shown below

       4-vault: '{"KMS_PROVIDER":"vaulttokens","KMS_SERVICE_NAME":"vault","VAULT_ADDR":"https://vault.qe.rh-ocs.com:8200","VAULT_BACKEND_PATH":"bug","VAULT_CACERT":"ocs-kms-ca-secret- 
       35i1ma","VAULT_TLS_SERVER_NAME":"","VAULT_CLIENT_CERT":"ocs-kms-client-cert-ja7ani","VAULT_CLIENT_KEY":"ocs-kms-client-key-jrtckg","VAULT_NAMESPACE":"","VAULT_TOKEN_NAME":"ocs-kms- 
       token","VAULT_CACERT_FILE":"fullchain.pem","VAULT_CLIENT_CERT_FILE":"cert.pem","VAULT_CLIENT_KEY_FILE":"privkey.pem","VAULT_BACKEND":"kv"}'

5. Save the configmap
6. Create a PVC using the SC and attach it to a pod
7. Check the status of the PVC and Pod


Actual results:

The PV creation was successful, however the pod creation failed with the error given below and no key was created in Vault.

Warning  FailedMapVolume         8s (x5 over 18s)  kubelet                  MapVolume.SetUpDevice failed for volume "pvc-56732d4c-6203-4f9b-b2a8-0728a8f00135" : rpc error: code = Internal desc = failed to encrypt rbd image ocs-storagecluster-cephblockpool/csi-vol-8986b89d-f8f9-11eb-972f-0a580a810211: failed parsing data for get passphrase request for 0001-0011-openshift-storage-0000000000000001-8986b89d-f8f9-11eb-972f-0a580a810211

Expected results:
Either the error message should be improved to reflect the actual cause of the failure or we can prevent VAULT_BACKEND value to override the auto-detection value in situations like this, if possible

Additional info:
- When the values are switched, i.e., set VAULT_BACKEND to "kv-v2" for a backend path using kv, the PVC and pod creation succeeds. However the key in vault is created under the subpath /data inside the backend path

       3-vault: '{"KMS_PROVIDER":"vaulttokens","KMS_SERVICE_NAME":"vault","VAULT_ADDR":"https://vault.qe.rh-ocs.com:8200","VAULT_BACKEND_PATH":"bug1","VAULT_CACERT":"ocs-kms-ca-secret 
       o1icyg","VAULT_TLS_SERVER_NAME":"","VAULT_CLIENT_CERT":"ocs-kms-client-cert-h0z11d","VAULT_CLIENT_KEY":"ocs-kms-client-key-hgy4x","VAULT_NAMESPACE":"","VAULT_TOKEN_NAME":"ocs-kms- 
       token","VAULT_CACERT_FILE":"fullchain.pem","VAULT_CLIENT_CERT_FILE":"cert.pem","VAULT_CLIENT_KEY_FILE":"privkey.pem","VAULT_BACKEND":"kv-v2"}'

       $ vault kv list bug1/data
       Keys
       ----
       0001-0011-openshift-storage-0000000000000001-dd67e191-f8de-11eb-972f-0a580a810211
       0001-0011-openshift-storage-0000000000000001-f638f218-f8f9-11eb-972f-0a580a810211

- The same error should be observed in OCS 4.8 for PVC creation, since the key is generated in vault during PV creation in 4.8


Note You need to log in before you can comment on or make changes to this bug.