Bug 2074558

Summary:	rook osd prepare failed with this error - failed to set kek as an environment variable: key encryption key is empty
Product:	[Red Hat Storage] Red Hat OpenShift Data Foundation	Reporter:	Sunil Kumar Acharya <sheggodu>
Component:	rook	Assignee:	Sébastien Han <shan>
Status:	CLOSED ERRATA	QA Contact:	Neha Berry <nberry>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	4.10	CC:	agantony, ebenahar, kramdoss, madam, muagarwa, ocs-bugs, odf-bz-bot, shan, tal.yichye, tnielsen
Target Milestone:	---
Target Release:	ODF 4.10.1
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Previously, the OpenShift Data Foundation deployment failed as the encryption code was triggered when it was not activated in an IBM environment. This happened because the IBM environment variables were present in the pods. With this update, there is better detection of the cluster-wide encryption scenarios, and the encryption code is activated as you desire.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-05-11 16:03:41 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Sunil Kumar Acharya 2022-04-12 13:41:42 UTC

This bug was initially created as a copy of Bug #2073920

I am copying this bug because: 



Description of problem (please be detailed as possible and provide log
snippests):

We installed the Redhat ODF operator from the console and try to create the storagesystem for ibm flashsystem with no encryption option.
The osd pods were not created and the rook-ceph-osd-prepare pods are stuck in CrashLoopBaskOff with the following error:

022-04-06 11:55:37.199155 E | op-osd: failed to provision OSD(s) on PVC ocs-deviceset-ibm-odf-test-0-data-0jmkhn. &{OSDs:[] Status:failed PvcBackedOSD:true Message:failed to set kek as an environment variable: key encryption key is empty}


Version of all relevant components (if applicable):

RH ODF operator: Image quay.io/rhceph-dev/ocs-registry:4.10.0-211
OCP version: 4.10.3

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?

yes, we can't install our new ibm odf operator

Is there any workaround available to the best of your knowledge?

no

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?

1

Can this issue reproducible?
 yes

Can this issue reproduce from the UI?
yes

If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. Change redhat-operator catalog source to use quay.io/rhceph-dev/ocs-registry:4.10.0-211 image
2. Install odf operator for ocp 4.10 from the UI
3. Install StorageSystem ibm-flashsystem from the UI

Actual results:
rook-ceph-osd-prepare pods are stuck in CrashLoopBackOff with the kms error:

022-04-06 11:55:37.199155 E | op-osd: failed to provision OSD(s) on PVC ocs-deviceset-ibm-odf-test-0-data-0jmkhn. &{OSDs:[] Status:failed PvcBackedOSD:true Message:failed to set kek as an environment variable: key encryption key is empty}

Expected results:

successfully create the osd pods with no kms encryption

Additional info:

Comment 2 Travis Nielsen 2022-04-12 15:18:38 UTC

*** Bug 2074513 has been marked as a duplicate of this bug. ***

Comment 11 Sébastien Han 2022-05-04 09:37:54 UTC

"because IBM environment variables were present"

Otherwise LGTM, thanks!

Comment 16 Tal Yichye 2022-05-10 06:50:05 UTC

Hi,

After installing with ODF 4.10.1 image, this issue does not appear any more.
The OSDs prepare jobs are now in state 'in progress' and no OSDs pods were deployed - I opened another BZ ticket as Sébastien requested, to investigate the new issue https://bugzilla.redhat.com/show_bug.cgi?id=2081431

Thanks

Comment 17 Elad 2022-05-10 07:21:09 UTC

Moving to VERIFIED based on comment #16

Comment 22 errata-xmlrpc 2022-05-11 16:03:41 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenShift Data Foundation 4.10.1 Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:2182