Bug 2223575
| Summary: | Pod odf-operator-controller-manager CrashLoopBackOff | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat OpenShift Data Foundation | Reporter: | Aviad Polak <apolak> |
| Component: | odf-operator | Assignee: | Nitin Goyal <nigoyal> |
| Status: | CLOSED ERRATA | QA Contact: | Elad <ebenahar> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 4.14 | CC: | amagrawa, muagarwa, odf-bz-bot, vavuthu |
| Target Milestone: | --- | Keywords: | Automation, Regression |
| Target Release: | ODF 4.14.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | 4.14.0-77 | Doc Type: | No Doc Update |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2023-11-08 18:52:48 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
*** Bug 2225007 has been marked as a duplicate of this bug. *** Please add qa_ack Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.14.0 security, enhancement & bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2023:6832 The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days |
Description of problem (please be detailed as possible and provide log snippests): Cluster is running idle for few days (~5d21h) pod odf-operator-controller-manager is in CrashLoopBackOff and 1643 restarts panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1312890] goroutine 1 [running]: k8s.io/client-go/discovery.convertAPIResource(...) /remote-source/app/vendor/k8s.io/client-go/discovery/aggregated_discovery.go:88 k8s.io/client-go/discovery.convertAPIGroup({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc000049620, 0x15}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...}) /remote-source/app/vendor/k8s.io/client-go/discovery/aggregated_discovery.go:69 +0x570 k8s.io/client-go/discovery.SplitGroupsAndResources({{{0xc000048690, 0x15}, {0xc000115340, 0x1b}}, {{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...}) /remote-source/app/vendor/k8s.io/client-go/discovery/aggregated_discovery.go:35 +0x118 k8s.io/client-go/discovery.(*DiscoveryClient).downloadAPIs(0x990014?) /remote-source/app/vendor/k8s.io/client-go/discovery/discovery_client.go:310 +0x47c k8s.io/client-go/discovery.(*DiscoveryClient).GroupsAndMaybeResources(0x1316ad3?) /remote-source/app/vendor/k8s.io/client-go/discovery/discovery_client.go:198 +0x5c k8s.io/client-go/discovery.ServerGroupsAndResources({0x1d15618, 0xc0001270e0}) /remote-source/app/vendor/k8s.io/client-go/discovery/discovery_client.go:392 +0x59 k8s.io/client-go/discovery.(*DiscoveryClient).ServerGroupsAndResources.func1() /remote-source/app/vendor/k8s.io/client-go/discovery/discovery_client.go:356 +0x25 k8s.io/client-go/discovery.withRetries(0x2, 0xc0007f7080) /remote-source/app/vendor/k8s.io/client-go/discovery/discovery_client.go:621 +0x71 k8s.io/client-go/discovery.(*DiscoveryClient).ServerGroupsAndResources(0x0?) /remote-source/app/vendor/k8s.io/client-go/discovery/discovery_client.go:355 +0x3a k8s.io/client-go/restmapper.GetAPIGroupResources({0x1d15618?, 0xc0001270e0?}) /remote-source/app/vendor/k8s.io/client-go/restmapper/discovery.go:148 +0x42 sigs.k8s.io/controller-runtime/pkg/client/apiutil.NewDynamicRESTMapper.func1() /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/client/apiutil/dynamicrestmapper.go:94 +0x25 sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).setStaticMapper(...) /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/client/apiutil/dynamicrestmapper.go:130 sigs.k8s.io/controller-runtime/pkg/client/apiutil.NewDynamicRESTMapper(0xc0004f0480?, {0x0, 0x0, 0xb3af44060a3b5201?}) /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/client/apiutil/dynamicrestmapper.go:110 +0x182 sigs.k8s.io/controller-runtime/pkg/cluster.setOptionsDefaults.func1(0xc000218d20?) /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/cluster/cluster.go:217 +0x25 sigs.k8s.io/controller-runtime/pkg/cluster.New(0xc0003246c0, {0xc00010fa28, 0x1, 0x0?}) /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/cluster/cluster.go:159 +0x18d sigs.k8s.io/controller-runtime/pkg/manager.New(_, {0xc000218d20, 0x0, 0x0, {{0x1d11ce0, 0xc00047f580}, 0x0}, 0x1, {0x0, 0x0}, ...}) /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/manager/manager.go:359 +0xf9 main.main() /remote-source/app/main.go:98 +0x4a3 Version of all relevant components (if applicable): odf: full_version: 4.14.0-61 clientVersion: buildDate: "2023-07-05T14:18:17Z" compiler: gc gitCommit: 1a0e89316 gitTreeState: dirty gitVersion: v4.2.0-alpha.0-1949-g1a0e893 goVersion: go1.20.3 major: "" minor: "" platform: darwin/arm64 kustomizeVersion: v5.0.1 openshiftVersion: 4.14.0-0.nightly-2023-07-11-092038 releaseClientVersion: 4.14.0-0.ci-2023-07-11-133509 serverVersion: buildDate: "2023-07-07T16:22:44Z" compiler: gc gitCommit: ee9c1a1f13b06f5e2a79dcbd06285ec3f8315448 gitTreeState: clean gitVersion: v1.27.3+af29f64 goVersion: go1.20.3 major: "1" minor: "27" platform: linux/amd64 Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? yes Is there any workaround available to the best of your knowledge? no, as far as i know Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? 1 Steps to Reproduce: 1.run cluster for few days, no specific action taken 2. 3. Additional info: noobaa with same issue: noobaa-operator-5cdf6c975-464tn 1/2 CrashLoopBackOff 2303