Bug 2187602

Summary: Storagecluster is stuck in Progressing state with FIPS enabled
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: narayanspg <ngowda>
Component: Multi-Cloud Object GatewayAssignee: Nimrod Becker <nbecker>
Status: CLOSED NOTABUG QA Contact: krishnaram Karthick <kramdoss>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.13CC: lmauda, muagarwa, ocs-bugs, odf-bz-bot, pbalogh, sasundar
Target Milestone: ---Keywords: TestBlocker
Target Release: ---   
Hardware: ppc64le   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-04-23 07:40:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
noobaa logs none

Description narayanspg 2023-04-18 06:27:17 UTC 
Description of problem (please be detailed as possible and provide log
snippests):
This BZ is in continuation to BZ (noobaa pods crashing) https://bugzilla.redhat.com/show_bug.cgi?id=2175612

storagecluster is stuck in progressing state with reason - "Waiting on Nooba instance to finish initialization" . all pods are up and running 

Created OCP cluster with FIPS enabled. Deployed ODF through UI.

When we are create OCP cluster with FIPS disabled then ODF deployment is getting successful. 

Version of all relevant components (if applicable):
OCP 4.13 
[root@fips5-cicd-odf-e564-sao01-bastion-0 ~]# oc describe csv odf-operator.v4.13.0 -n openshift-storage | grep full
Labels:       full_version=4.13.0-130
          f:full_version:

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
with FIPS enabled we are not able to deploy ODF

Is there any workaround available to the best of your knowledge?
NO

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
3

Can this issue reproducible?
Yes

Can this issue reproduce from the UI?
Yes

If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. Create OCP cluster with FIPS enabled
2. deploy ODF
3. storagecluster is not getting to ready state


Actual results:
storagecluster is not getting to ready state

Expected results:
storagecluster should get to ready state

Additional info:
[root@fips5-cicd-odf-e564-sao01-bastion-0 ~]# oc logs  noobaa-core-0 | grep fips
Apr-10 10:07:56.195 [/20]   [LOG] CONSOLE:: detect_fips_mode: found /proc/sys/crypto/fips_enabled with value 1
detect_fips_mode: found /proc/sys/crypto/fips_enabled with value 1
detect_fips_mode: found /proc/sys/crypto/fips_enabled with value 1
detect_fips_mode: found /proc/sys/crypto/fips_enabled with value 1
[root@fips5-cicd-odf-e564-sao01-bastion-0 ~]# oc get storagecluster
NAME                 AGE    PHASE         EXTERNAL   CREATED AT             VERSION
ocs-storagecluster   179m   Progressing              2023-04-10T10:03:25Z   4.13.0
[root@fips5-cicd-odf-e564-sao01-bastion-0 ~]# oc describe storagecluster ocs-storagecluster | grep Waiting
    Message:               Waiting on Nooba instance to finish initialization
[root@fips5-cicd-odf-e564-sao01-bastion-0 ~]#
Comment 2 narayanspg 2023-04-18 06:32:48 UTC 
Created attachment 1957934 [details]
noobaa logs
Comment 4 Liran Mauda 2023-04-20 05:46:37 UTC 
Petr and I tried to reproduce it on none ppc64le cluster.
We cannot reproduce.

Narayanaswamy, is it still happening in ppc64le? 

if it does, we need a cluster, as we can't see the RC in the logs.

Best Regards,
Liran.
Comment 6 narayanspg 2023-04-21 06:24:31 UTC 
with FIPS enabled on latest build ODF is working fine.

[root@fips7-cicd-odf-e25f-sao01-bastion-0 ~]# oc get clusterversion
NAME      VERSION                                      AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.13.0-0.nightly-ppc64le-2023-04-19-155600   True        False         14h     Cluster version is 4.13.0-0.nightly-ppc64le-2023-04-19-155600

[root@fips7-cicd-odf-e25f-sao01-bastion-0 ~]# oc describe csv odf-operator.v4.13.0 -n openshift-storage | grep full
Labels:       full_version=4.13.0-170
          f:full_version:
[root@fips7-cicd-odf-e25f-sao01-bastion-0 ~]#

[root@fips7-cicd-odf-e25f-sao01-bastion-0 ~]# oc logs -f noobaa-core-0 | grep fips
Apr-21 6:12:40.605 [/20] [LOG] CONSOLE:: detect_fips_mode: found /proc/sys/crypto/fips_enabled with value 1
detect_fips_mode: found /proc/sys/crypto/fips_enabled with value 1

[root@fips7-cicd-odf-e25f-sao01-bastion-0 ~]# oc get storagecluster
NAME AGE PHASE EXTERNAL CREATED AT VERSION
ocs-storagecluster 6m21s Ready 2023-04-21T06:09:16Z 4.13.0
Comment 7 narayanspg 2023-04-21 06:25:08 UTC 
Please close this BZ. Thank you.