Bug 2223575 - Pod odf-operator-controller-manager CrashLoopBackOff
Summary: Pod odf-operator-controller-manager CrashLoopBackOff
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenShift Data Foundation
Classification: Red Hat Storage
Component: odf-operator
Version: 4.14
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ODF 4.14.0
Assignee: Nitin Goyal
QA Contact: Elad
URL:
Whiteboard:
: 2225007 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-07-18 10:07 UTC by Aviad Polak
Modified: 2024-03-08 04:25 UTC (History)
4 users (show)

Fixed In Version: 4.14.0-77
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-11-08 18:52:48 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github red-hat-storage odf-operator pull 330 0 None open godeps: update k8s godeps to v0.26.4 2023-07-18 14:02:12 UTC
Red Hat Product Errata RHSA-2023:6832 0 None None None 2023-11-08 18:53:54 UTC

Description Aviad Polak 2023-07-18 10:07:06 UTC
Description of problem (please be detailed as possible and provide log
snippests):
Cluster is running idle for few days (~5d21h)
pod odf-operator-controller-manager is in CrashLoopBackOff and 1643 restarts

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1312890]

goroutine 1 [running]:
k8s.io/client-go/discovery.convertAPIResource(...)
	/remote-source/app/vendor/k8s.io/client-go/discovery/aggregated_discovery.go:88
k8s.io/client-go/discovery.convertAPIGroup({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc000049620, 0x15}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})
	/remote-source/app/vendor/k8s.io/client-go/discovery/aggregated_discovery.go:69 +0x570
k8s.io/client-go/discovery.SplitGroupsAndResources({{{0xc000048690, 0x15}, {0xc000115340, 0x1b}}, {{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})
	/remote-source/app/vendor/k8s.io/client-go/discovery/aggregated_discovery.go:35 +0x118
k8s.io/client-go/discovery.(*DiscoveryClient).downloadAPIs(0x990014?)
	/remote-source/app/vendor/k8s.io/client-go/discovery/discovery_client.go:310 +0x47c
k8s.io/client-go/discovery.(*DiscoveryClient).GroupsAndMaybeResources(0x1316ad3?)
	/remote-source/app/vendor/k8s.io/client-go/discovery/discovery_client.go:198 +0x5c
k8s.io/client-go/discovery.ServerGroupsAndResources({0x1d15618, 0xc0001270e0})
	/remote-source/app/vendor/k8s.io/client-go/discovery/discovery_client.go:392 +0x59
k8s.io/client-go/discovery.(*DiscoveryClient).ServerGroupsAndResources.func1()
	/remote-source/app/vendor/k8s.io/client-go/discovery/discovery_client.go:356 +0x25
k8s.io/client-go/discovery.withRetries(0x2, 0xc0007f7080)
	/remote-source/app/vendor/k8s.io/client-go/discovery/discovery_client.go:621 +0x71
k8s.io/client-go/discovery.(*DiscoveryClient).ServerGroupsAndResources(0x0?)
	/remote-source/app/vendor/k8s.io/client-go/discovery/discovery_client.go:355 +0x3a
k8s.io/client-go/restmapper.GetAPIGroupResources({0x1d15618?, 0xc0001270e0?})
	/remote-source/app/vendor/k8s.io/client-go/restmapper/discovery.go:148 +0x42
sigs.k8s.io/controller-runtime/pkg/client/apiutil.NewDynamicRESTMapper.func1()
	/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/client/apiutil/dynamicrestmapper.go:94 +0x25
sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).setStaticMapper(...)
	/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/client/apiutil/dynamicrestmapper.go:130
sigs.k8s.io/controller-runtime/pkg/client/apiutil.NewDynamicRESTMapper(0xc0004f0480?, {0x0, 0x0, 0xb3af44060a3b5201?})
	/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/client/apiutil/dynamicrestmapper.go:110 +0x182
sigs.k8s.io/controller-runtime/pkg/cluster.setOptionsDefaults.func1(0xc000218d20?)
	/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/cluster/cluster.go:217 +0x25
sigs.k8s.io/controller-runtime/pkg/cluster.New(0xc0003246c0, {0xc00010fa28, 0x1, 0x0?})
	/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/cluster/cluster.go:159 +0x18d
sigs.k8s.io/controller-runtime/pkg/manager.New(_, {0xc000218d20, 0x0, 0x0, {{0x1d11ce0, 0xc00047f580}, 0x0}, 0x1, {0x0, 0x0}, ...})
	/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/manager/manager.go:359 +0xf9
main.main()
	/remote-source/app/main.go:98 +0x4a3

Version of all relevant components (if applicable):
odf: full_version: 4.14.0-61
clientVersion:
  buildDate: "2023-07-05T14:18:17Z"
  compiler: gc
  gitCommit: 1a0e89316
  gitTreeState: dirty
  gitVersion: v4.2.0-alpha.0-1949-g1a0e893
  goVersion: go1.20.3
  major: ""
  minor: ""
  platform: darwin/arm64
kustomizeVersion: v5.0.1
openshiftVersion: 4.14.0-0.nightly-2023-07-11-092038
releaseClientVersion: 4.14.0-0.ci-2023-07-11-133509
serverVersion:
  buildDate: "2023-07-07T16:22:44Z"
  compiler: gc
  gitCommit: ee9c1a1f13b06f5e2a79dcbd06285ec3f8315448
  gitTreeState: clean
  gitVersion: v1.27.3+af29f64
  goVersion: go1.20.3
  major: "1"
  minor: "27"
  platform: linux/amd64


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
yes

Is there any workaround available to the best of your knowledge?
no, as far as i know

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
1


Steps to Reproduce:
1.run cluster for few days, no specific action taken
2.
3.



Additional info:
noobaa with same issue:
noobaa-operator-5cdf6c975-464tn                                   1/2     CrashLoopBackOff   2303

Comment 4 Nitin Goyal 2023-07-24 09:58:22 UTC
*** Bug 2225007 has been marked as a duplicate of this bug. ***

Comment 5 Mudit Agarwal 2023-07-25 07:08:23 UTC
Please add qa_ack

Comment 12 errata-xmlrpc 2023-11-08 18:52:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.14.0 security, enhancement & bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6832

Comment 13 Red Hat Bugzilla 2024-03-08 04:25:57 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.