Bug 2232101

Summary:	[ODF-4.13] Default backingstore stuck at Connecting with OCP 4.14
Product:	[Red Hat Storage] Red Hat OpenShift Data Foundation	Reporter:	Daniel Osypenko <dosypenk>
Component:	Multi-Cloud Object Gateway	Assignee:	Shira Dymnik <sdymnik>
Status:	ASSIGNED ---	QA Contact:	krishnaram Karthick <kramdoss>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.13	CC:	dzaken, odf-bz-bot
Target Milestone:	---	Flags:	dzaken: needinfo? (dosypenk)
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Daniel Osypenko 2023-08-15 09:56:26 UTC

Description of problem (please be detailed as possible and provide log
snippests):

[ODF-4.13] Default backingstore stuck at Connecting with OCP 4.14

RPC: connection closed while request is pending
        wss://noobaa-mgmt.openshift-storage.svc.cluster.local:443/rpc/
        wss://noobaa-mgmt.openshift-storage.svc.cluster.local:443/rpc/-1
      reason: TemporaryError

oc get backingstore -n openshift-storage                 
NAME                           TYPE            PHASE        AGE
noobaa-default-backing-store   s3-compatible   Connecting   45h


full yaml -> https://url.corp.redhat.com/backing-store-not-ready

Also unable to create OBC using noobaa-default-backing-store (OBC in status pending)

Version of all relevant components (if applicable):


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?


Is there any workaround available to the best of your knowledge?


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible?
yes

Can this issue reproduce from the UI?
yes

If this is a regression, please provide more details to justify this:
regression, older versions and version above (ODF 4.14) Backing store works OK

Steps to Reproduce:
1. Deploy cluster with OCP 4.14, ODF 4.13, leave it for ~6 hours
2. Check the default backing store state `oc get backingstore -n openshift-storage `
3.


Actual results:

noobaa-default-backing-store - status ready

Expected results:
noobaa-default-backing-store - status connecting

Additional info:
must-gather: https://url.corp.redhat.com/must-gather-backing-store-Connecting

Comment 3 Daniel Osypenko 2023-08-16 08:50:33 UTC

hello Shira. Thank you for detailed analyses. I have deployed another OCP 4.14 ODF 4.13 cluster and for now it does not have problem described in bz. The problem has been seen once.