Bug 2089552

Summary: [MS v2] Cannot create StorageClassClaim
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Jilju Joy <jijoy>
Component: ocs-operatorAssignee: Jose A. Rivera <jarrpa>
Status: CLOSED ERRATA QA Contact: Jilju Joy <jijoy>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.11CC: madam, mrajanna, muagarwa, nberry, ocs-bugs, odf-bz-bot, sostapov
Target Milestone: ---Keywords: TestBlocker
Target Release: ODF 4.11.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 4.11.0-81 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-08-24 13:53:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jilju Joy 2022-05-24 02:24:36 UTC
Description of problem (please be detailed as possible and provide log
snippests):
Storageclassclaim is not getting created in a consumer cluster.

$ oc get storageclassclaim
NAME                            STORAGETYPE        PHASE
test-storageclassclaim-cephfs   sharedfilesystem   Configuring
test-storageclassclaim-rbd      blockpool          Configuring


Error in ocs-oparator logs. (same error for both Storage type)

{"level":"info","ts":1653318585.848355,"logger":"controller.storageclassclaim","msg":"Reconciling StorageClassClaim.","reconciler group":"ocs.openshift.io","reconciler kind":"StorageClassClaim","name":"test-storageclassclaim-cephfs","namespace":"openshift-storage","StorageClassClaim":"openshift-storage/test-storageclassclaim-cephfs"}
{"level":"info","ts":1653318585.8484063,"logger":"controller.storageclassclaim","msg":"Running StorageClassClaim controller in Consumer Mode","reconciler group":"ocs.openshift.io","reconciler kind":"StorageClassClaim","name":"test-storageclassclaim-cephfs","namespace":"openshift-storage","StorageClassClaim":"openshift-storage/test-storageclassclaim-cephfs"}
{"level":"error","ts":1653318585.8703942,"logger":"controller.storageclassclaim","msg":"Reconciler error","reconciler group":"ocs.openshift.io","reconciler kind":"StorageClassClaim","name":"test-storageclassclaim-cephfs","namespace":"openshift-storage","error":"failed to initiate fulfillment of StorageClassClaim: rpc error: code = Internal desc = failed to fulfill storage class claim for \"76a7e6a8-5e98-4cc0-ba7a-d7135cff1ce9\". failed to create a StorageClassClaim named \"storageclassclaim-f937e4648711fe9e6e0af6d856befd72\" for consumer \"76a7e6a8-5e98-4cc0-ba7a-d7135cff1ce9\" and claim \"test-storageclassclaim-cephfs\". storageclassclaims.ocs.openshift.io \"storageclassclaim-f937e4648711fe9e6e0af6d856befd72\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/...


Template used to create storageclass claim:

apiVersion: ocs.openshift.io/v1alpha1
kind: StorageClassClaim
metadata:
  name: test-storageclassclaim-cephfs
spec:
  type: sharedfilesystem


must-gather logs: http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/jijoy-m23-c1/jijoy-m23-c1_20220523T084457/logs/testcases_1653320023/
=======================================================================
Version of all relevant components (if applicable):
$ oc get csv
NAME                                      DISPLAY                       VERSION           REPLACES                                  PHASE
mcg-operator.v4.11.0                      NooBaa Operator               4.11.0            mcg-operator.v4.10.2                      Succeeded
ocs-operator.v4.11.0                      OpenShift Container Storage   4.11.0            ocs-operator.v4.10.2                      Succeeded
ocs-osd-deployer.v2.0.2                   OCS OSD Deployer              2.0.2             ocs-osd-deployer.v2.0.1                   Succeeded
odf-csi-addons-operator.v4.11.0           CSI Addons                    4.11.0            odf-csi-addons-operator.v4.10.2           Succeeded
odf-operator.v4.11.0                      OpenShift Data Foundation     4.11.0            odf-operator.v4.10.2                      Succeeded
ose-prometheus-operator.4.10.0            Prometheus Operator           4.10.0            ose-prometheus-operator.4.8.0             Succeeded
route-monitor-operator.v0.1.418-6459408   Route Monitor Operator        0.1.418-6459408   route-monitor-operator.v0.1.408-c2256a2   Succeeded


$ oc get csv odf-operator.v4.11.0 -o yaml | grep full_version
    full_version: 4.11.0-75


OCP 4.10.13


====================================================================

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
Yes. Cannot create StorageClassClaim.

Is there any workaround available to the best of your knowledge?
No

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
1

Can this issue reproducible?
Yes

Can this issue reproduce from the UI?


If this is a regression, please provide more details to justify this:
This is a new feature

Steps to Reproduce:
1. Create StorageClassClaim on consumer cluster using the yaml given below.
apiVersion: ocs.openshift.io/v1alpha1
kind: StorageClassClaim
metadata:
  name: test-storageclassclaim-cephfs
spec:
  type: sharedfilesystem

2. Verify the phase of the StorageClassClaim


Actual results:
Phase of StorageClassClaim is always "Configuring".

Expected results:
StorageClassClaim creation should be successful.

Additional info:

Comment 5 Jilju Joy 2022-06-01 11:16:04 UTC
Getting this error when creating storageclassclaim in openshift-storage namespace.
$ oc get storageclassclaim -n openshift-storage
NAME                             STORAGETYPE        PHASE
test-storageclassclaim-cephfs1   sharedfilesystem   Configuring

ocs-operator logs:

{"level":"info","ts":1654078402.6853268,"logger":"controller.storageclassclaim","msg":"Reconciling StorageClassClaim.","reconciler group":"ocs.openshift.io","reconciler kind":"StorageClassClaim","name":"test-storageclassclaim-cephfs1","namespace":"openshift-storage","StorageClassClaim":"openshift-storage/test-storageclassclaim-cephfs1"}
{"level":"info","ts":1654078402.6853898,"logger":"controller.storageclassclaim","msg":"Running StorageClassClaim controller in Consumer Mode","reconciler group":"ocs.openshift.io","reconciler kind":"StorageClassClaim","name":"test-storageclassclaim-cephfs1","namespace":"openshift-storage","StorageClassClaim":"openshift-storage/test-storageclassclaim-cephfs1"}
{"level":"error","ts":1654078402.7412593,"logger":"controller.storageclassclaim","msg":"Reconciler error","reconciler group":"ocs.openshift.io","reconciler kind":"StorageClassClaim","name":"test-storageclassclaim-cephfs1","namespace":"openshift-storage","error":"failed to get StorageClassClaim config: rpc error: code = Unavailable desc = storage class claim \"test-storageclassclaim-cephfs1\" for \"56f2103b-264f-45cf-887d-edb7c2bffaae\" is in \"Creating\" phase","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}


must-gather logs - http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/jijoy-j1-c1/jijoy-j1-c1_20220601T043509/logs/testcases_1654079214/

The Phase of the storage class claim is the same as reported initially in this bug. The error is different. Workaround is to respin ocs-operator pods on consumer and provider. Opened a new bug https://bugzilla.redhat.com/show_bug.cgi?id=2092372

Comment 6 Jilju Joy 2022-06-09 12:17:32 UTC
Marking this as verified because the current issue in creating storageclassclaim is different from the issue initially reported in the bug.
Creation of storageclassclaim is tested again as described in https://bugzilla.redhat.com/show_bug.cgi?id=2092372#c5


Tested in version:

ocs-operator.v4.11.0 
ocs-osd-deployer.v2.0.2
odf-csi-addons-operator.v4.11.0
odf-operator.v4.11.0  
ODF full version 4.11.0-90
OCP 4.10.16

Comment 9 errata-xmlrpc 2022-08-24 13:53:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.11.0 security, enhancement, & bugfix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:6156