Bug 1941526

Summary:	controller-manager-operator: Observed a panic: nil pointer dereference
Product:	OpenShift Container Platform	Reporter:	Jan Chaloupka <jchaloup>
Component:	openshift-controller-manager	Assignee:	Corey Daley <cdaley>
openshift-controller-manager sub component:	controller-manager	QA Contact:	wewang <wewang>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	high
Priority:	medium	CC:	aos-bugs
Version:	4.8
Target Milestone:	---
Target Release:	4.8.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-07-27 22:54:36 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1944142

Description Jan Chaloupka 2021-03-22 10:24:40 UTC

https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_machine-config-operator/2422/pull-ci-openshift-machine-config-operator-master-e2e-aws-workers-rhel7/1373002979191296000

```
pods/openshift-controller-manager-operator_openshift-controller-manager-operator-5b986dc7cc-2f75c_openshift-controller-manager-operator.log.gz:E0319 21:09:24.129164       1 runtime.go:76] Observed a panic: runtime error: invalid memory address or nil pointer dereference
```

https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_machine-config-operator/2422/pull-ci-openshift-machine-config-operator-master-e2e-aws-workers-rhel7/1373002979191296000/artifacts/e2e-aws-workers-rhel7/gather-extra/artifacts/pods/openshift-controller-manager-operator_openshift-controller-manager-operator-5b986dc7cc-2f75c_openshift-controller-manager-operator.log:

```
E0319 21:09:24.129164       1 runtime.go:76] Observed a panic: runtime error: invalid memory address or nil pointer dereference
goroutine 12063 [running]:
k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1.1(0xc0013f9920)
	k8s.io/apiserver.1-rc.0/pkg/server/filters/timeout.go:106 +0x113
panic(0x1fcb900, 0x34fd8c0)
	runtime/panic.go:969 +0x1b9
k8s.io/apiserver/plugin/pkg/authorizer/webhook.(*WebhookAuthorizer).Authorize(0xc000b02540, 0x2635ce0, 0xc0009ecff0, 0x2654740, 0xc0014f77c0, 0x8, 0xc001632d00, 0x16745af, 0x26362a0, 0xc0014882c0)
	k8s.io/apiserver.1-rc.0/plugin/pkg/authorizer/webhook/webhook.go:208 +0x8b9
k8s.io/apiserver/pkg/authorization/union.unionAuthzHandler.Authorize(0xc0007ca150, 0x1, 0x1, 0x2635ce0, 0xc0009ecff0, 0x2654740, 0xc0014f77c0, 0x1, 0x3, 0xc0011fcda0, ...)
	k8s.io/apiserver.1-rc.0/pkg/authorization/union/union.go:52 +0xfe
k8s.io/apiserver/pkg/authorization/union.unionAuthzHandler.Authorize(0xc0007649e0, 0x2, 0x2, 0x2635ce0, 0xc0009ecff0, 0x2654740, 0xc0014f77c0, 0x25abb88, 0x1e9a7e0, 0xc0013642c0, ...)
	k8s.io/apiserver.1-rc.0/pkg/authorization/union/union.go:52 +0xfe
k8s.io/apiserver/pkg/endpoints/filters.WithAuthorization.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filters/authorization.go:59 +0x165
net/http.HandlerFunc.ServeHTTP(0xc0009ffb40, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:71 +0x186
net/http.HandlerFunc.ServeHTTP(0xc0009ffb80, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/server/filters.WithMaxInFlightLimit.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/server/filters/maxinflight.go:184 +0x4cf
net/http.HandlerFunc.ServeHTTP(0xc0007d7da0, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:95 +0x165
net/http.HandlerFunc.ServeHTTP(0xc0007d7dd0, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filters.WithImpersonation.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filters/impersonation.go:50 +0x23dd
net/http.HandlerFunc.ServeHTTP(0xc0009ffbc0, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:71 +0x186
net/http.HandlerFunc.ServeHTTP(0xc0009ffc00, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:95 +0x165
net/http.HandlerFunc.ServeHTTP(0xc0007d7e00, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:71 +0x186
net/http.HandlerFunc.ServeHTTP(0xc0009ffc40, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:95 +0x165
net/http.HandlerFunc.ServeHTTP(0xc0007d7e60, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filters.WithAuthentication.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f000)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filters/authentication.go:70 +0x6d2
net/http.HandlerFunc.ServeHTTP(0xc0004ec0a0, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f000)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107ef00)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:80 +0x38a
net/http.HandlerFunc.ServeHTTP(0xc0009ffc80, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107ef00)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1(0xc0013f9920, 0xc000772100, 0x2636360, 0xc0001f6ed0, 0xc00107ef00)
	k8s.io/apiserver.1-rc.0/pkg/server/filters/timeout.go:111 +0xb8
created by k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP
	k8s.io/apiserver.1-rc.0/pkg/server/filters/timeout.go:97 +0x1cc
```

result of result.Status at https://github.com/openshift/cluster-openshift-controller-manager-operator/blob/master/vendor/k8s.io/apiserver/plugin/pkg/authorizer/webhook/webhook.go#L208 is nil based on the panic trace. Most likely webhook.WithExponentialBackoff returns non-nil error which is not handled. Making result empty.

Fixed by https://github.com/kubernetes/kubernetes/pull/97820. Will get resolved during the rebase to 1.21. Worth backporting to older 4.X versions.

Comment 8 errata-xmlrpc 2021-07-27 22:54:36 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438