Description of problem (please be detailed as possible and provide log snippests): GCP IPI platform, build: ocs-registry:4.15.0-155 Resource ocs-storagecluster is in phase: Progressing Version of all relevant components (if applicable): ocs-registry:4.15.0-155 Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? Yes Is there any workaround available to the best of your knowledge? NA Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? 1/1 Can this issue reproducible? not sure Can this issue reproduce from the UI? Not tried If this is a regression, please provide more details to justify this: Yes Steps to Reproduce: 1. install odf using ocs-ci on GCP platform 2. check storagecluster satus 3. Actual results: Status: Conditions: Last Heartbeat Time: 2024-03-04T15:14:30Z Last Transition Time: 2024-03-04T15:14:30Z Message: Version check successful Reason: VersionMatched Status: False Type: VersionMismatch Last Heartbeat Time: 2024-03-04T15:20:55Z Last Transition Time: 2024-03-04T15:18:57Z Message: Reconcile completed successfully Reason: ReconcileCompleted Status: True Type: ReconcileComplete Last Heartbeat Time: 2024-03-04T15:18:10Z Last Transition Time: 2024-03-04T15:18:10Z Message: Reconcile completed successfully Reason: ReconcileCompleted Status: True Type: Available Last Heartbeat Time: 2024-03-04T15:20:55Z Last Transition Time: 2024-03-04T15:18:11Z Message: Waiting on Nooba instance to finish initialization Reason: NoobaaInitializing Status: True Type: Progressing Last Heartbeat Time: 2024-03-04T15:18:10Z Last Transition Time: 2024-03-04T15:14:30Z Message: Reconcile completed successfully Reason: ReconcileCompleted Status: False Type: Degraded Last Heartbeat Time: 2024-03-04T15:19:15Z Last Transition Time: 2024-03-04T15:18:11Z Message: CephCluster is creating: Processing OSD 2 on PVC "ocs-deviceset-1-data-0g67hr" Reason: ClusterStateCreating Status: False Type: Upgradeable Current Mon Count: 3 Expected results: storagecluster should be in Ready state Additional info: noobaa operator log: 2024-03-04T15:20:51.229604636Z time="2024-03-04T15:20:51Z" level=info msg="✅ Exists: Secret \"noobaa-gcp-bucket-creds\"\n" 2024-03-04T15:20:51.232068718Z panic: reflect.Value.Interface: cannot return value obtained from unexported field or method [recovered] 2024-03-04T15:20:51.232068718Z panic: reflect.Value.Interface: cannot return value obtained from unexported field or method 2024-03-04T15:20:51.232068718Z 2024-03-04T15:20:51.232068718Z goroutine 2413 [running]: 2024-03-04T15:20:51.232068718Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1() 2024-03-04T15:20:51.232068718Z /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.3/pkg/internal/controller/controller.go:116 +0x1fa 2024-03-04T15:20:51.232096516Z panic({0x22946c0, 0x2d73ae0}) 2024-03-04T15:20:51.232096516Z /usr/lib/golang/src/runtime/panic.go:884 +0x213 2024-03-04T15:20:51.232096516Z reflect.valueInterface({0x2608920?, 0xc001ccdbc0?, 0xc001eed470?}, 0x66?) 2024-03-04T15:20:51.232109747Z /usr/lib/golang/src/reflect/value.go:1491 +0xd8 2024-03-04T15:20:51.232109747Z reflect.Value.Interface(...) 2024-03-04T15:20:51.232118542Z /usr/lib/golang/src/reflect/value.go:1480 2024-03-04T15:20:51.232126381Z github.com/noobaa/noobaa-operator/v5/pkg/system.(*Reconciler).prepareGCPBackingStore(0xc000832280) 2024-03-04T15:20:51.232134578Z /remote-source/app/pkg/system/phase4_configuring.go:1034 +0x765 2024-03-04T15:20:51.232134578Z github.com/noobaa/noobaa-operator/v5/pkg/system.(*Reconciler).ReconcileDefaultBackingStore(0xc000832280) 2024-03-04T15:20:51.232143559Z /remote-source/app/pkg/system/phase4_configuring.go:720 +0x35a 2024-03-04T15:20:51.232152147Z github.com/noobaa/noobaa-operator/v5/pkg/system.(*Reconciler).ReconcilePhaseConfiguring(0xc000832280) 2024-03-04T15:20:51.232152147Z /remote-source/app/pkg/system/phase4_configuring.go:80 +0x12e 2024-03-04T15:20:51.232171402Z github.com/noobaa/noobaa-operator/v5/pkg/system.(*Reconciler).ReconcilePhases(0x2815542?) 2024-03-04T15:20:51.232171402Z /remote-source/app/pkg/system/reconciler.go:547 +0x65 2024-03-04T15:20:51.232171402Z github.com/noobaa/noobaa-operator/v5/pkg/system.(*Reconciler).Reconcile(0xc000832280) 2024-03-04T15:20:51.232171402Z /remote-source/app/pkg/system/reconciler.go:422 +0x33b 2024-03-04T15:20:51.232181675Z github.com/noobaa/noobaa-operator/v5/pkg/controller/noobaa.Add.func1({0xc001370ce0?, 0x40dd8a?}, {{{0xc00110c3f0?, 0x30?}, {0xc0015fe7d6?, 0x25affa0?}}}) 2024-03-04T15:20:51.232198620Z /remote-source/app/pkg/controller/noobaa/noobaa_controller.go:53 +0xe5 2024-03-04T15:20:51.232207649Z sigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile(0x7f6c21bc61d8?, {0x2d9a008?, 0xc0020ca300?}, {{{0xc00110c3f0?, 0x0?}, {0xc0015fe7d6?, 0xc001370d48?}}}) 2024-03-04T15:20:51.232240234Z /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.3/pkg/reconcile/reconcile.go:111 +0x43 2024-03-04T15:20:51.232240234Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x2d9eb48?, {0x2d9a008?, 0xc0020ca300?}, {{{0xc00110c3f0?, 0xb?}, {0xc0015fe7d6?, 0x0?}}}) 2024-03-04T15:20:51.232272791Z /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.3/pkg/internal/controller/controller.go:119 +0xc8 2024-03-04T15:20:51.232272791Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000acb540, {0x2d99f60, 0xc0002f16d0}, {0x24d5d00?, 0xc00105b6a0?}) 2024-03-04T15:20:51.232290081Z /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.3/pkg/internal/controller/controller.go:316 +0x3ca 2024-03-04T15:20:51.232298206Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000acb540, {0x2d99f60, 0xc0002f16d0}) 2024-03-04T15:20:51.232306292Z /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.3/pkg/internal/controller/controller.go:266 +0x1d9 2024-03-04T15:20:51.232314545Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2() 2024-03-04T15:20:51.232314545Z /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.3/pkg/internal/controller/controller.go:227 +0x85 2024-03-04T15:20:51.232322429Z created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 2024-03-04T15:20:51.232322429Z /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.3/pkg/internal/controller/controller.go:223 +0x587 job: https://url.corp.redhat.com/4bca560 must gather: https://url.corp.redhat.com/aae2e6b
Update ======== build: ocs-registry:4.15.0-158 Didn't see the panic in build 158 ( out of 2 times, first time deployment is successfull and second hit the https://bugzilla.redhat.com/show_bug.cgi?id=2269379 ) job: https://url.corp.redhat.com/21f8765 logs: https://url.corp.redhat.com/1033e76 2024-03-14 12:14:22 06:44:22 - MainThread - ocs_ci.ocs.resources.storage_cluster - INFO - Check if StorageCluster: ocs-storagecluster is in Succeeded phase 2024-03-14 12:14:22 06:44:22 - MainThread - ocs_ci.utility.utils - INFO - Executing command: oc --kubeconfig /home/jenkins/current-cluster-dir/openshift-cluster-dir/auth/kubeconfig -n openshift-storage get StorageCluster ocs-storagecluster -n openshift-storage -o yaml 2024-03-14 12:14:22 06:44:22 - MainThread - ocs_ci.ocs.ocp - INFO - Resource ocs-storagecluster is in phase: Ready! Marking as Verified
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.15.0 security, enhancement, & bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2024:1383