Bug 1975272 - [RFE] [KMS] Add support for auto-detection of the Vault KV version
Summary: [RFE] [KMS] Add support for auto-detection of the Vault KV version
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenShift Data Foundation
Classification: Red Hat Storage
Component: rook
Version: 4.8
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ODF 4.9.0
Assignee: Sébastien Han
QA Contact: Rachael
URL:
Whiteboard:
Depends On:
Blocks: 2011326
TreeView+ depends on / blocked
 
Reported: 2021-06-23 11:24 UTC by Rachael
Modified: 2023-08-09 17:03 UTC (History)
11 users (show)

Fixed In Version: v4.9.0-158.ci
Doc Type: Enhancement
Doc Text:
.Auto-detection of Vault's Secret Key/Value store version With this enhancement, Vault's Secret Key/Value store version is auto-detected.
Clone Of:
Environment:
Last Closed: 2021-12-13 17:44:31 UTC
Embargoed:
prasriva: needinfo-


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github red-hat-storage rook pull 17 0 None None None 2021-09-10 06:42:03 UTC
Github red-hat-storage rook pull 303 0 None open Bug 1975272: ceph: fix kms auto-detection when full TLS 2021-10-18 07:24:55 UTC
Github rook rook pull 8265 0 None open ceph: auto detect vault k/v version 2021-07-05 16:18:09 UTC
Github rook rook pull 8618 0 None None None 2021-08-30 16:30:42 UTC
Github rook rook pull 8867 0 None open ceph: fix kms auto-detection when full TLS 2021-09-29 13:34:58 UTC
Red Hat Product Errata RHSA-2021:5086 0 None None None 2021-12-13 17:44:50 UTC

Description Rachael 2021-06-23 11:24:37 UTC
Description of problem (please be detailed as possible and provide log
snippets):

With OCS 4.7.2 (https://bugzilla.redhat.com/show_bug.cgi?id=1970583) and OCS 4.8 (https://bugzilla.redhat.com/show_bug.cgi?id=1936858), the kv-v2 secret engine API is now supported for cluster-wide encryption with KMS. In order to successfully deploy a cluster in this case, the VAULT_BACKEND parameter has to be set to v2 in the ocs-kms-connection-details configmap. 

   $ oc get cm ocs-kms-connection-details -o yaml
   apiVersion: v1
   data:
     KMS_PROVIDER: vault
     KMS_SERVICE_NAME: vault
     VAULT_ADDR: https://vault.qe.rh-ocs.com:8200
     VAULT_BACKEND: v2
     VAULT_BACKEND_PATH: test-kv2
     VAULT_CACERT: ocs-kms-ca-secret-znu27r
     VAULT_CLIENT_CERT: ocs-kms-client-cert-7od4d
     VAULT_CLIENT_KEY: ocs-kms-client-key-8obbs
     VAULT_NAMESPACE: ocs
     VAULT_TLS_SERVER_NAME: vault.qe.rh-ocs.com

This is currently done by editing the configmap once it is created during the storagecluster creation. It would be good to have an option in the UI, which will allow the user to set this parameter while configuring the vault details in the storagecluster creation page, instead of having to edit it in the configmap directly.

Version of all relevant components (if applicable):
OCP: 4.8.0-0.nightly-2021-06-19-005119
OCS: ocs-operator.v4.8.0-424.ci

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
Yes, without the UI option available, the deployment would not succeed in such a case unless the configmap is not edited manually.

Is there any workaround available to the best of your knowledge?
Yes, edit the ocs-kms-connection-details configmap to add the parameter

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
3

Can this issue reproducible?
Yes

Can this issue reproduce from the UI?
Yes

If this is a regression, please provide more details to justify this:
No

Steps to Reproduce:
Steps to Reproduce:
1. Create a backend path in Vault with kv-v2
   $ vault secrets enable -path=test-kv2 kv-v2
   Success! Enabled the kv-v2 secrets engine at: test-kv2/

2. Enter the path created above when deploying OCS with cluster wide encryption using KMS enabled in UI

3. Check the status of the OSD pods

Actual results:
The OSD pods don't come up and fail with the following error, since the VAULT_BACKEND parameter is not set to v2:

$ oc logs  rook-ceph-osd-0-54f9f6c6c-hrnw2 -c encryption-kms-get-kek
["Invalid path for a versioned K/V secrets engine. See the API docs for the appropriate API endpoints to use. If using the Vault CLI, use 'vault kv get' for this operation."] 

Expected results:
The UI should have an option to set VAULT_BACKEND value, to avoid deployment failure

Comment 3 Mudit Agarwal 2021-06-24 03:11:19 UTC
No RFE in 4.8 atm.

Comment 4 Nishanth Thomas 2021-06-24 15:25:01 UTC
Rfe, reducing the severity to medium

Comment 5 Sanjal Katiyar 2021-07-02 09:16:02 UTC
As per the discussion: https://chat.google.com/room/AAAA2G9_Elw/MMbfZrj_Vos moving the BZ to rook component.
Targeting for 4.9

Comment 8 Sébastien Han 2021-09-10 06:42:03 UTC
part of https://github.com/red-hat-storage/rook/pull/17

Comment 12 Sébastien Han 2021-09-28 09:27:49 UTC
Thanks Rachael, the problem is "VAULT_SECRET_ENGINE" set to transit and not "kv".

Jiffin, do you know why this is set this way?
IIRC, the transit should be part of the object store kms connection details, NOT part of the cephcluster KMS connection details.
Is this a UI bug?

Comment 13 Sébastien Han 2021-09-28 09:28:40 UTC
@Rachael in the meantime, can you force "VAULT_SECRET_ENGINE" to "kv" to validate the auto detection works? Thanks!

Comment 14 Jiffin 2021-09-28 09:46:15 UTC
AFAIR OCS-Op sets secret engines differently for OSD encryption and for RGW, IMO most likely a bug there. Looping Pranshu who worked on the OCS-Op related changes

Comment 17 Mudit Agarwal 2021-10-18 06:53:40 UTC
The latest PR needs to be backported to 4.9

Comment 18 Sébastien Han 2021-10-18 07:25:34 UTC
https://github.com/red-hat-storage/rook/pull/303

Comment 26 errata-xmlrpc 2021-12-13 17:44:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat OpenShift Data Foundation 4.9.0 enhancement, security, and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:5086


Note You need to log in before you can comment on or make changes to this bug.