Bug 1965898

Summary:	[RFE] : Read the passphrase for PV encryption from a different secret
Product:	[Red Hat Storage] Red Hat OpenShift Data Foundation	Reporter:	N Balachandran <nibalach>
Component:	csi-driver	Assignee:	Rakshith <rar>
Status:	CLOSED DEFERRED	QA Contact:	Rachael <rgeorge>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	4.8	CC:	ebenahar, etamir, madam, mrajanna, muagarwa, nberry, ndevos, ocs-bugs, odf-bz-bot, omitrani, rar
Target Milestone:	---	Keywords:	FutureFeature
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	4.9.0-51.ci	Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-01-25 09:36:41 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description N Balachandran 2021-05-31 04:54:17 UTC

Description of problem (please be detailed as possible and provide log
snippests):

CephCSI provides an option to encrypt PVs where the DEK is encrypted with the passphrase and stored in the rbd metadata.
This passphrase is read from the secrets provided in the csi.storage.k8s.io/node-stage-secret-name and csi.storage.k8s.io/provisioner-secret-name.
These secrets are reconciled by Rook and cannot be modified by the user.



Version of all relevant components (if applicable):


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?

The OCS Dedicated Global Service intends to support PV encryption. If a user does not provide his own key, OCS dedicated will generate a passphrase per user to encrypt his PVs. 
This passphrase needs to be saved in a secret which the CSI component will read and use to encrypt the PVs. 
It cannot be saved in the secrets currently used by CSI as Rook will reconcile them and remove the passphrase.


Changes requested:
Allow the passphrase to be saved in a separate secret  which the Ceph CSI can then read and use for PV encryption. This would be similar to the approach used for AWS KMS encryption.








Actual results:


Expected results:


Additional info:

Comment 3 Niels de Vos 2021-05-31 07:43:28 UTC

Currently, Ceph-CSI has a mechanism to use configure a SecretsMetadataKMS type that stores the DEK in the metadata of the RBD-image:
- https://github.com/ceph/ceph-csi/blob/81809500be8dde7760978b3daa2e15bc2740eec1/examples/kms/vault/csi-kms-connection-details.yaml#L34-L37

This KMS 'service' uses the `encryptionPassphrase` that is optionally configured together with the Ceph cluster details (created by Rook).

We can extend the SecretsMetadataKMS configuration with an optional set of parameters (Namespace and Secret name) that contain the passphrase to encrypt/decrypt the DEK from the RBD-image metadata.

Comment 4 N Balachandran 2021-05-31 09:08:10 UTC

(In reply to Niels de Vos from comment #3)
> Currently, Ceph-CSI has a mechanism to use configure a SecretsMetadataKMS
> type that stores the DEK in the metadata of the RBD-image:
> -
> https://github.com/ceph/ceph-csi/blob/
> 81809500be8dde7760978b3daa2e15bc2740eec1/examples/kms/vault/csi-kms-
> connection-details.yaml#L34-L37
> 
> This KMS 'service' uses the `encryptionPassphrase` that is optionally
> configured together with the Ceph cluster details (created by Rook).
> 
> We can extend the SecretsMetadataKMS configuration with an optional set of
> parameters (Namespace and Secret name) that contain the passphrase to
> encrypt/decrypt the DEK from the RBD-image metadata.

That sounds ideal. And that would mean that we could have different passphrases configured for different StorageClasses, correct?

Comment 5 Rakshith 2021-07-14 09:53:01 UTC

The following changes have been made for this RFE 

Encryption metadata configuration
CephCSI can generate unique passphrase (DEK Data-Encryption-Key) for each volume to be used to encrypt/decrypt data. The passphrase (DEK) is encrypted by encryptionPassphrase (KEK Key-Encryption-Key) and stored in the image metadata of the volume.

To encrypt rbd volumes with metadata encryption, users need to set encrypted: "true" and encryptionKMSID to a unique identifier in storageclass. This unique identifier should be similar to the examples. The configuration must include "encryptionKMSType": "metadata". The encryptionPassphrase is fetched based on the following conditions:

if "secretName" key is specified, encryptionPassphrase is fetched from this secret and "secretNamespace" value is used for namespace if specified else Tenant/Kubernetes namespace (i.e., namespace where the PVC was created) is used.
if "secretName" key is not specified, encryptionPassphrase is fetched from storageclass secrets csi.storage.k8s.io/provisioner-secret-namespace / csi.storage.k8s.io/provisioner-secret-name and csi.storage.k8s.io/node-stage-secret-namespace / csi.storage.k8s.io/node-stage-secret-name similar to the previous Encryption Configuration.

More details can be found here
https://github.com/ceph/ceph-csi/blob/devel/docs/deploy-rbd.md#encryption-metadata-configuration

Comment 6 Rakshith 2021-08-16 09:10:22 UTC

Steps to use user secret based metadata encryption.

https://hackmd.io/FwvH3IWMQjuS1tdHTKDiuA?view

Comment 8 Mudit Agarwal 2021-09-01 14:28:27 UTC

Moving this to 4.10 as this was not a part of 4.9 planning, from QE side only regression runs are enough for 4.9

Nithya, will the dedicated team give it a try as far as testing is concerned?

Comment 9 N Balachandran 2021-09-01 15:08:31 UTC

(In reply to Mudit Agarwal from comment #8)
> Moving this to 4.10 as this was not a part of 4.9 planning, from QE side
> only regression runs are enough for 4.9
> 
> Nithya, will the dedicated team give it a try as far as testing is concerned?

@Ohad, can you take a look at this?

Comment 10 Ohad 2021-10-13 06:17:12 UTC

The OSF-MS team has shifted focus to a solution that does not includes PV encryption for the duration of the OCP 4.10 development cycle (PV encryption got scoped out).
And as such we do not have the platform to test this under correct conditions.

Comment 11 Niels de Vos 2021-10-15 08:46:30 UTC

This is already available in current builds, just waiting on validation.

Comment 12 Elad 2021-10-25 09:51:50 UTC

@Niels, @Eran,

What are the expectations here with regards to verification?
The bug is devel backed for 4.10.0 but 'fixed in version' is 4.9.0-51.ci. 
As described by Rachael in comment #7, this is an RFE for using kubernetes secrets instead of KMS, which was never supposed to be supported or planned for 4.9.

Are we expected to verify based on regression testing in 4.9?

Comment 15 Elad 2021-12-14 12:55:43 UTC

Thanks Eran

Comment 17 Niels de Vos 2022-01-18 08:55:48 UTC

(In reply to Ohad from comment #10)
> The OSF-MS team has shifted focus to a solution that does not includes PV
> encryption for the duration of the OCP 4.10 development cycle (PV encryption
> got scoped out).
> And as such we do not have the platform to test this under correct
> conditions.

Ohad, or Nithya, is this something that you would like to see supported in a future version of ODF?

If not, this feature request can be closed. The functionality is available in upstream Ceph-CSI, but there does not need to be support for it in the ODF product.