Bug 2178502 - ODF operator fails to create openshift-storage.noobaa.io StorageClass
Summary: ODF operator fails to create openshift-storage.noobaa.io StorageClass
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat OpenShift Data Foundation
Classification: Red Hat Storage
Component: Multi-Cloud Object Gateway
Version: 4.10
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Nimrod Becker
QA Contact: krishnaram Karthick
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-03-15 06:14 UTC by Soumi Mitra
Modified: 2023-08-09 16:49 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-03-21 07:10:08 UTC
Embargoed:


Attachments (Terms of Use)

Description Soumi Mitra 2023-03-15 06:14:41 UTC
Description of problem (please be detailed as possible and provide log
snippests):

1] While installation Openshift or during upgrade OCP/ODF, customer observed that noobaa deployment in incomplete state or noobaa storageclass is not created
2] This was seen in several instances by customer
3] To resolve the issue customer reinstalled the cluster and in this case support proposed customer to redeploy noobaa following https://access.redhat.com/solutions/5948631
4] Customer mentioned they do not want a one time solution by redeploying noobaa in ODF following the steps/ procedure rather wants to understand why the deployment end up in this state with noobaa deployment being incomplete and possible fix for it.

5] Looking into the noobaa-operator logs, it logged 'Response Error: Code=UNAUTHORIZED Message ' . The "Code=UNAUTHORIZED Message=account not found" messages are seen due to account information missing from the noobaa-db. The noobaa operator is unable to find the required details in the db.


2023-03-06T06:34:24.817709639Z time="2023-03-06T06:34:24Z" level=error msg="⚠️  RPC: account.read_account() Response Error: Code=UNAUTHORIZED Message=not anonymous method read_account"
2023-03-06T06:34:24.817709639Z time="2023-03-06T06:34:24Z" level=error msg="ReconcileObject: Error Secret  cannot read admin account info, error: not anonymous method read_account" sys=openshift-storage/noobaa
2023-03-06T06:34:24.817709639Z time="2023-03-06T06:34:24Z" level=info msg="SetPhase: temporary error during phase \"Configuring\"" sys=openshift-storage/noobaa
2023-03-06T06:34:24.817742895Z time="2023-03-06T06:34:24Z" level=warning msg="<U+23F3> Temporary Error: cannot read admin account info, error: not anonymous method read_account" sys=openshift-storage/noobaa



From customer

+++++++

we are not interested how we can fix ODF after cluster deployment. We just want a first time right installation of ODF. Especially in a DR situation where recovery time is crucial we don't want to waste time fixing ODF or redeploying the cluster hoping ODF will work correctly.

We have re(deployed) 3 different clusters in the past months I think all have exhibited this problem at one stage or another. This cluster was just the latest example.

- he wants to understand why ODF can't bootstrap itself sometimes
- if it is something with their environment - they need to understand how to fix it, to make deployment more reliable
- if it is ODF bug, he wants a fix eventually.

++++++

Version of all relevant components (if applicable):

- OCP version running is 4.10.52 and ODF version is 4.10.10
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.52   True        False         3d17h   Cluster version is 4.10.52

csv

NAME                                      DISPLAY                                    VERSION   REPLACES                                  PHASE

mcg-operator.v4.10.10                     NooBaa Operator                            4.10.10   mcg-operator.v4.10.9                      Succeeded
namespace-configuration-operator.v1.2.4   Namespace Configuration Operator           1.2.4     namespace-configuration-operator.v1.2.3   Succeeded
ocs-operator.v4.10.10                     OpenShift Container Storage                4.10.10   ocs-operator.v4.10.9                      Succeeded
odf-csi-addons-operator.v4.10.10          CSI Addons                                 4.10.10   odf-csi-addons-operator.v4.10.9           Succeeded
odf-operator.v4.10.10                     OpenShift Data Foundation                  4.10.10   odf-operator.v4.10.9                      Succeeded



Does this issue impact your ability to continue to work with the product
Yes, customer plan to use OBC for their application so this impact their cluster


Is there any workaround available to the best of your knowledge?
Yes, to rebuild noobaa https://access.redhat.com/solutions/5948631

Rate from 1 - 5 the complexity of the scenario you performed that caused this
4


Can this issue reproducible?
Yes

Can this issue reproduce from the UI?
No

If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. New install of OCP followed by ODF deployment
2. ODF upgrade



Actual results:

Noobaa deployment is incomplete


Expected results:
Noobaa deployment should finish with ODF deployment

Additional info:


Regards,
Soumi


Note You need to log in before you can comment on or make changes to this bug.