Bug 2182037

Summary: [ODF to ODF on ROSA] Storageclassclaims in Configuring state
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Jilju Joy <jijoy>
Component: odf-managed-serviceAssignee: Ohad <omitrani>
Status: CLOSED NOTABUG QA Contact: Neha Berry <nberry>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.13CC: mrajanna, ocs-bugs, odf-bz-bot
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-03-30 08:32:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jilju Joy 2023-03-27 11:33:38 UTC
Description of problem:
As part of testing the feature "Enable deployment of ODF product on ROSA (to enable managed service)", it was observed that the deafult storageclassclaims on the consumer cluster was not ready but in "Configuring" phase.

$  oc get storageclassclaim -A
NAMESPACE     NAME                          STORAGETYPE        PHASE
odf-storage   ocs-storagecluster-ceph-rbd   blockpool          Configuring
odf-storage   ocs-storagecluster-cephfs     sharedfilesystem   Configuring


Storagecluster is  created in "odf-storage" namespace.

Error logs in ocs-operator-844bd4b4b4-z64bz pod:

{"level":"info","ts":"2023-03-23T11:50:30Z","msg":"Reconciling StorageClassClaim.","controller":"storageclassclaim","controllerGroup":"ocs.openshift.io","controllerKind":"StorageClassClaim","StorageClassClaim":{"name":"ocs-storagecluster-cephfs","namespace":"odf-storage"},"namespace":"odf-storage","name":"ocs-storagecluster-cephfs","reconcileID":"160d778f-b454-4a38-b31a-b3965fe06709","StorageClassClaim":"odf-storage/ocs-storagecluster-cephfs"}
{"level":"info","ts":"2023-03-23T11:50:30Z","msg":"Running StorageClassClaim controller in Consumer Mode","controller":"storageclassclaim","controllerGroup":"ocs.openshift.io","controllerKind":"StorageClassClaim","StorageClassClaim":{"name":"ocs-storagecluster-cephfs","namespace":"odf-storage"},"namespace":"odf-storage","name":"ocs-storagecluster-cephfs","reconcileID":"160d778f-b454-4a38-b31a-b3965fe06709","StorageClassClaim":"odf-storage/ocs-storagecluster-cephfs"}
{"level":"error","ts":"2023-03-23T11:50:30Z","msg":"Reconciler error","controller":"storageclassclaim","controllerGroup":"ocs.openshift.io","controllerKind":"StorageClassClaim","StorageClassClaim":{"name":"ocs-storagecluster-cephfs","namespace":"odf-storage"},"namespace":"odf-storage","name":"ocs-storagecluster-cephfs","reconcileID":"160d778f-b454-4a38-b31a-b3965fe06709","error":"failed to get StorageClassClaim config: rpc error: code = Unavailable desc = storage class claim \"ocs-storagecluster-cephfs\" for \"6da8aa7b-b093-4a85-a8e9-fcc19303be71\" is in \"Initializing\" phase","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:235"}


Adding must-gather logs . This does not contain logs from the namespace odf-storage because currently must-gather doesnt collect logs from custom namespaces. This request to enhance must-gather will be tracked seperately.

Consumer must-gather
http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/jijoy-m23-c1/jijoy-m23-c1_20230323T063536/logs/testcases_1679568559/

Provider must-gather http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/jijoy-m23-pr/jijoy-m23-pr_20230323T063512/logs/testcases_1679568584/

==================================================================================

Version-Release number of selected component (if applicable):
OCP 4.11.31

$ oc get csv
NAME                                               DISPLAY                           VERSION             REPLACES                                           PHASE
configure-alertmanager-operator.v0.1.516-bdea4ea   configure-alertmanager-operator   0.1.516-bdea4ea     configure-alertmanager-operator.v0.1.510-cd37263   Succeeded
mcg-operator.v4.13.0-109.stable                    NooBaa Operator                   4.13.0-109.stable                                                      Succeeded
observability-operator.v0.0.20                     Observability Operator            0.0.20              observability-operator.v0.0.19                     Succeeded
ocs-operator.v4.13.0-109.stable                    OpenShift Container Storage       4.13.0-109.stable                                                      Succeeded
odf-csi-addons-operator.v4.13.0-109.stable         CSI Addons                        4.13.0-109.stable                                                      Succeeded
odf-operator.v4.13.0-109.stable                    OpenShift Data Foundation         4.13.0-109.stable                                                      Succeeded
route-monitor-operator.v0.1.489-7d9fe90            Route Monitor Operator            0.1.489-7d9fe90     route-monitor-operator.v0.1.487-f6fa14d            Succeeded

=============================================================================
How reproducible:
1/1

Steps to Reproduce:
1. Create ODF to ODF provider and consumer clusters on ROSA.
Install storagecluster on a namespace other than openshift-storage. eg: odf-storage 
2. Check the default storageclassclaims in consumer.


Actual results:

The phase of the storageclassclaims "ocs-storagecluster-ceph-rbd" and "ocs-storagecluster-ceph-rbd" is "Configuring".

Expected results:
The phase of the storageclassclaims "ocs-storagecluster-ceph-rbd" and "ocs-storagecluster-ceph-rbd" should be "Ready".


Additional info: