Bug 2051661 - [GSS] Unable to Access Object Buckets [Noobaa BackingStore In Rejected Phase]
Summary: [GSS] Unable to Access Object Buckets [Noobaa BackingStore In Rejected Phase]
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Red Hat OpenShift Data Foundation
Classification: Red Hat Storage
Component: ceph
Version: 4.7
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Matt Benjamin (redhat)
QA Contact: Neha Berry
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-02-07 18:00 UTC by Craig Wayman
Modified: 2023-08-09 16:37 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-10-19 06:04:57 UTC
Embargoed:


Attachments (Terms of Use)
Errors (99.66 KB, image/png)
2022-02-07 18:00 UTC, Craig Wayman
no flags Details

Description Craig Wayman 2022-02-07 18:00:18 UTC
Created attachment 1859622 [details]
Errors

Description of problem (please be detailed as possible and provide log
snippets): 

  The customer began experiencing MCG issues and was seeing the error of "Backing store mode: INITIALIZING" after an incremental update within OCS v4.6.

  The customer was informed to upgrade to OCS v4.7 which should've fixed the issue (referenced Bug 1911266). Currently, the customer is viewing numerous errors and is unable to access previously established buckets. The following  events/errors are displayed:

NOT ENOUGH HEALTH RESOURCES

Buckets are in Error States

Service is Degraded

Backing store mode: ALL_NODES_OFFLINE

Warning  BackingStorePhaseRejected


InvalidBucketState: The request is not valid with the current state of the bucket. status code: 409

Version of all relevant components (if applicable):
OCS 4.6 and after upgrade to OCS 4.7 issue still persists.

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?

Yes, the customer does not have access to data in previously established buckets. 

Is there any workaround available to the best of your knowledge?

No, Tried restoring Multicloud-Object-Gateway IAW Chapter 10 (4.7 OCS Troubleshooting Product Doc). i.e. Restarted all the pods related to the MCG and restarted the RADOS Object Gateway (RGW) pod.

After performing the steps to restore MCG, the customer is now able to create buckets but still cannot access the previous buckets.

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?



Can this issue reproducible?

No


Can this issue reproduce from the UI?

No


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1.
2.
3.


Actual results:


Expected results:


Additional info:

Comment 3 Jiffin 2022-03-31 05:44:46 UTC
can you please attach the must gather or logs to see the status of rgw pods and service. There is very limited info for debugging this issue

Comment 12 Craig Wayman 2022-06-20 14:10:16 UTC
After being provided with the information in case comment #85, the customer has closed the case. The issue seems to be resolved, we can close this BZ. 

Regards,


Craig Wayman
TSE Red Hat OpenShift Data Foundations (ODF) 
Customer Experience and Engagement, NA


Note You need to log in before you can comment on or make changes to this bug.