Bug 2274392

Summary:	PV encryption with AZURE KMS failed with certificate error.
Product:	[Red Hat Storage] Red Hat OpenShift Data Foundation	Reporter:	Parag Kamble <pakamble>
Component:	csi-driver	Assignee:	Praveen M <mpraveen>
Status:	CLOSED ERRATA	QA Contact:	krishnaram Karthick <kramdoss>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.16	CC:	kramdoss, mpraveen, ndevos, odf-bz-bot
Target Milestone:	---	Flags:	kramdoss: needinfo+
Target Release:	ODF 4.16.0
Hardware:	All
OS:	Unspecified
Whiteboard:
Fixed In Version:	4.16.0-84	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2024-07-17 13:19:04 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Parag Kamble 2024-04-10 19:32:19 UTC

Description of problem (please be detailed as possible and provide log
snippests):

PV encryption with AZURE KMS failed with certificate error. Following error is getting while creation of PVC and its remain in Pending state.

failed to provision volume with StorageClass "ocs-storagecluster-ceph-rbd-encrypted": rpc error: code = InvalidArgument desc = invalid encryption kms configuration: failed to decode client certificate: illegal base64 data at input byte 0


Version of all relevant components (if applicable): 4.16


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?


Is there any workaround available to the best of your knowledge? 


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible? Yes


Can this issue reproduce from the UI? Yes


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. Start Deployment of ODF 4.16 on Azure platform
2. In storage cluster configuration phase select storageclass encryption with AZURE key vault.
3. Provide all the Azure Key vault related parameters and upload the certificate  file. 
4. Complete the ODF installation.
5. Verify storageclass "ocs-storagecluster-ceph-rbd-encrypted" is created successfully.
6. Create a PVC using "ocs-storagecluster-ceph-rbd-encrypted" Storageclass.
7. Wait till PVC is in Bound state.


Actual results:
PVC remains in Pending state and its popup certificate Error.

Expected results:
PVC should be in Bound state.

Additional info:

Cluster INformation
-=-=-=-=-=-=-=-=-=-=-
> ocs get csv
NAME                                        DISPLAY                            VERSION            REPLACES   PHASE
mcg-operator.v4.16.0-73.stable              NooBaa Operator                    4.16.0-73.stable              Succeeded
ocs-client-operator.v4.16.0-73.stable       OpenShift Data Foundation Client   4.16.0-73.stable              Succeeded
ocs-operator.v4.16.0-73.stable              OpenShift Container Storage        4.16.0-73.stable              Succeeded
odf-csi-addons-operator.v4.16.0-73.stable   CSI Addons                         4.16.0-73.stable              Succeeded
odf-operator.v4.16.0-73.stable              OpenShift Data Foundation          4.16.0-73.stable              Succeeded
odf-prometheus-operator.v4.16.0-73.stable   Prometheus Operator                4.16.0-73.stable              Succeeded
rook-ceph-operator.v4.16.0-73.stable        Rook-Ceph                          4.16.0-73.stable              Succeeded

> ocs get cm ocs-kms-connection-details -o yaml
apiVersion: v1
data:
  AZURE_CERT_SECRET_NAME: azure-ocs-rs9n6gzm
  AZURE_CLIENT_ID: ec78e481-8052-4ba1-b01d-ce5a47827ab5
  AZURE_TENANT_ID: 9cf78105-e3e9-4321-b88d-b001b66c762b
  AZURE_VAULT_URL: https://ocsqe-azure-kv.vault.azure.net/
  KMS_PROVIDER: azure-kv
  KMS_SERVICE_NAME: azure-connection
kind: ConfigMap
metadata:
  creationTimestamp: "2024-04-10T11:07:48Z"
  name: ocs-kms-connection-details
  namespace: openshift-storage
  resourceVersion: "81134"
  uid: 72ce09f5-bad1-4dfe-bbf9-3c73e97b13c5

Comment 8 Niels de Vos 2024-04-15 15:08:35 UTC

Downsteam backport https://github.com/red-hat-storage/ceph-csi/pull/294 is waiting on missing qa_ack+, pm_ack+ and a "Target Release" before getting merged.

Comment 15 Praveen M 2024-05-08 07:11:48 UTC

The issue was never exposed to the customers and is fixed in this release.

Comment 16 errata-xmlrpc 2024-07-17 13:19:04 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.16.0 security, enhancement & bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2024:4591