Description of problem (please be detailed as possible and provide log
snippests):

vSphere Disconnected mode

storagecluster is in Progressing state due to Noobaa instance fails to finish initialization


Version of all relevant components (if applicable):

openshift installer (4.16.0-0.nightly-2024-06-19-153641)
ocs-registry:4.16.0-129


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
Yes

Is there any workaround available to the best of your knowledge?
No

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
1

Can this issue reproducible?
1/1

Can this issue reproduce from the UI?


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. install ODF using disconnected mode
2. check storagecluster status
3.


Actual results:

storage cluster status

Status:
  Conditions:
    Last Heartbeat Time:      2024-06-19T23:08:05Z
    Last Transition Time:     2024-06-19T23:08:05Z
    Message:                  Version check successful
    Reason:                   VersionMatched
    Status:                   False
    Type:                     VersionMismatch
    Last Heartbeat Time:      2024-06-19T23:14:38Z
    Last Transition Time:     2024-06-19T23:11:00Z
    Message:                  Reconcile completed successfully
    Reason:                   ReconcileCompleted
    Status:                   True
    Type:                     ReconcileComplete
    Last Heartbeat Time:      2024-06-19T23:08:05Z
    Last Transition Time:     2024-06-19T23:08:05Z
    Message:                  Initializing StorageCluster
    Reason:                   Init
    Status:                   False
    Type:                     Available
    Last Heartbeat Time:      2024-06-19T23:14:38Z
    Last Transition Time:     2024-06-19T23:08:05Z
    Message:                  Waiting on Nooba instance to finish initialization
    Reason:                   NoobaaInitializing
    Status:                   True
    Type:                     Progressing
    Last Heartbeat Time:      2024-06-19T23:08:05Z
    Last Transition Time:     2024-06-19T23:08:05Z
    Message:                  Initializing StorageCluster
    Reason:                   Init
    Status:                   False
    Type:                     Degraded
    Last Heartbeat Time:      2024-06-19T23:08:05Z
    Last Transition Time:     2024-06-19T23:08:05Z
    Message:                  Initializing StorageCluster
    Reason:                   Init
    Status:                   Unknown
    Type:                     Upgradeable
  Current Mon Count:          3
.
.
.
  Phase:  Progressing


Expected results:

Storage cluster should be in Ready state


Additional info:

noobaa operator log has below error

2024-06-19T23:04:21.753711860Z time="2024-06-19T23:04:21Z" level=info msg="operator-image: noobaa/noobaa-operator:5.16.0\n"
2024-06-19T23:04:57.423291561Z [controller-runtime] log.SetLogger(...) was never called; logs will not be displayed.
2024-06-19T23:04:57.423291561Z Detected at:
2024-06-19T23:04:57.423291561Z 	>  goroutine 1 [running]:
2024-06-19T23:04:57.423291561Z 	>  runtime/debug.Stack()
2024-06-19T23:04:57.423291561Z 	>  	/usr/lib/golang/src/runtime/debug/stack.go:24 +0x5e
2024-06-19T23:04:57.423291561Z 	>  sigs.k8s.io/controller-runtime/pkg/log.eventuallyFulfillRoot()
2024-06-19T23:04:57.423291561Z 	>  	/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.2/pkg/log/log.go:60 +0xcd
2024-06-19T23:04:57.423291561Z 	>  sigs.k8s.io/controller-runtime/pkg/log.(*delegatingLogSink).Enabled(0xc000857480, 0xede055b15?)
2024-06-19T23:04:57.423291561Z 	>  	/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.2/pkg/log/deleg.go:111 +0x32
2024-06-19T23:04:57.423291561Z 	>  github.com/go-logr/logr.Logger.Info({{0x2d6f770?, 0xc000857480?}, 0x2d78ee0?}, {0x26ced29, 0x18}, {0x0, 0x0, 0x0})
2024-06-19T23:04:57.423291561Z 	>  	/remote-source/deps/gomod/pkg/mod/github.com/go-logr/logr.1/logr.go:276 +0x72
2024-06-19T23:04:57.423291561Z 	>  github.com/operator-framework/operator-lib/leader.Become({0x2d6a7b0, 0x41db900}, {0x26c3664, 0x14}, {0x0, 0x0, 0x174ff54?})
2024-06-19T23:04:57.423291561Z 	>  	/remote-source/deps/gomod/pkg/mod/github.com/operator-framework/operator-lib.1-0.20240112142536-8e41bd5ce489/leader/leader.go:200 +0x9c5
2024-06-19T23:04:57.423291561Z 	>  github.com/noobaa/noobaa-operator/v5/pkg/operator.RunOperator(0xc0009dcc00, {0x41db900?, 0x0, 0x0})
2024-06-19T23:04:57.423291561Z 	>  	/remote-source/app/pkg/operator/manager.go:51 +0x12e
2024-06-19T23:04:57.423291561Z 	>  github.com/spf13/cobra.(*Command).execute(0xc0009dcc00, {0x41db900, 0x0, 0x0})
2024-06-19T23:04:57.423291561Z 	>  	/remote-source/deps/gomod/pkg/mod/github.com/spf13/cobra.0/command.go:987 +0xaa3
2024-06-19T23:04:57.423291561Z 	>  github.com/spf13/cobra.(*Command).ExecuteC(0xc000eb4000)
2024-06-19T23:04:57.423291561Z 	>  	/remote-source/deps/gomod/pkg/mod/github.com/spf13/cobra.0/command.go:1115 +0x3ff
2024-06-19T23:04:57.423291561Z 	>  github.com/spf13/cobra.(*Command).Execute(...)
2024-06-19T23:04:57.423291561Z 	>  	/remote-source/deps/gomod/pkg/mod/github.com/spf13/cobra.0/command.go:1039
2024-06-19T23:04:57.423291561Z 	>  github.com/noobaa/noobaa-operator/v5/pkg/cli.Run()
2024-06-19T23:04:57.423291561Z 	>  	/remote-source/app/pkg/cli/cli.go:61 +0x18
2024-06-19T23:04:57.423291561Z 	>  main.main()
2024-06-19T23:04:57.423291561Z 	>  	/remote-source/app/main.go:13 +0xf


job: https://url.corp.redhat.com/fc093d6
must gather logs: https://url.corp.redhat.com/f7c48c8