Bug 1941526 - controller-manager-operator: Observed a panic: nil pointer dereference
Summary: controller-manager-operator: Observed a panic: nil pointer dereference
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: openshift-controller-manager
Version: 4.8
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ---
: 4.8.0
Assignee: Corey Daley
QA Contact: wewang
URL:
Whiteboard:
Depends On:
Blocks: 1944142
TreeView+ depends on / blocked
 
Reported: 2021-03-22 10:24 UTC by Jan Chaloupka
Modified: 2021-07-27 22:54 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-27 22:54:36 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-openshift-controller-manager-operator pull 203 0 None open Bug 1941526: Bump kubernetes to 0.20.5 2021-03-26 13:33:03 UTC
Red Hat Product Errata RHSA-2021:2438 0 None None None 2021-07-27 22:54:57 UTC

Description Jan Chaloupka 2021-03-22 10:24:40 UTC
https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_machine-config-operator/2422/pull-ci-openshift-machine-config-operator-master-e2e-aws-workers-rhel7/1373002979191296000

```
pods/openshift-controller-manager-operator_openshift-controller-manager-operator-5b986dc7cc-2f75c_openshift-controller-manager-operator.log.gz:E0319 21:09:24.129164       1 runtime.go:76] Observed a panic: runtime error: invalid memory address or nil pointer dereference
```

https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_machine-config-operator/2422/pull-ci-openshift-machine-config-operator-master-e2e-aws-workers-rhel7/1373002979191296000/artifacts/e2e-aws-workers-rhel7/gather-extra/artifacts/pods/openshift-controller-manager-operator_openshift-controller-manager-operator-5b986dc7cc-2f75c_openshift-controller-manager-operator.log:

```
E0319 21:09:24.129164       1 runtime.go:76] Observed a panic: runtime error: invalid memory address or nil pointer dereference
goroutine 12063 [running]:
k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1.1(0xc0013f9920)
	k8s.io/apiserver.1-rc.0/pkg/server/filters/timeout.go:106 +0x113
panic(0x1fcb900, 0x34fd8c0)
	runtime/panic.go:969 +0x1b9
k8s.io/apiserver/plugin/pkg/authorizer/webhook.(*WebhookAuthorizer).Authorize(0xc000b02540, 0x2635ce0, 0xc0009ecff0, 0x2654740, 0xc0014f77c0, 0x8, 0xc001632d00, 0x16745af, 0x26362a0, 0xc0014882c0)
	k8s.io/apiserver.1-rc.0/plugin/pkg/authorizer/webhook/webhook.go:208 +0x8b9
k8s.io/apiserver/pkg/authorization/union.unionAuthzHandler.Authorize(0xc0007ca150, 0x1, 0x1, 0x2635ce0, 0xc0009ecff0, 0x2654740, 0xc0014f77c0, 0x1, 0x3, 0xc0011fcda0, ...)
	k8s.io/apiserver.1-rc.0/pkg/authorization/union/union.go:52 +0xfe
k8s.io/apiserver/pkg/authorization/union.unionAuthzHandler.Authorize(0xc0007649e0, 0x2, 0x2, 0x2635ce0, 0xc0009ecff0, 0x2654740, 0xc0014f77c0, 0x25abb88, 0x1e9a7e0, 0xc0013642c0, ...)
	k8s.io/apiserver.1-rc.0/pkg/authorization/union/union.go:52 +0xfe
k8s.io/apiserver/pkg/endpoints/filters.WithAuthorization.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filters/authorization.go:59 +0x165
net/http.HandlerFunc.ServeHTTP(0xc0009ffb40, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:71 +0x186
net/http.HandlerFunc.ServeHTTP(0xc0009ffb80, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/server/filters.WithMaxInFlightLimit.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/server/filters/maxinflight.go:184 +0x4cf
net/http.HandlerFunc.ServeHTTP(0xc0007d7da0, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:95 +0x165
net/http.HandlerFunc.ServeHTTP(0xc0007d7dd0, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filters.WithImpersonation.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filters/impersonation.go:50 +0x23dd
net/http.HandlerFunc.ServeHTTP(0xc0009ffbc0, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:71 +0x186
net/http.HandlerFunc.ServeHTTP(0xc0009ffc00, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:95 +0x165
net/http.HandlerFunc.ServeHTTP(0xc0007d7e00, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:71 +0x186
net/http.HandlerFunc.ServeHTTP(0xc0009ffc40, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:95 +0x165
net/http.HandlerFunc.ServeHTTP(0xc0007d7e60, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f700)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filters.WithAuthentication.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f000)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filters/authentication.go:70 +0x6d2
net/http.HandlerFunc.ServeHTTP(0xc0004ec0a0, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107f000)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1(0x7f749d1eb860, 0xc0001f6ed0, 0xc00107ef00)
	k8s.io/apiserver.1-rc.0/pkg/endpoints/filterlatency/filterlatency.go:80 +0x38a
net/http.HandlerFunc.ServeHTTP(0xc0009ffc80, 0x7f749d1eb860, 0xc0001f6ed0, 0xc00107ef00)
	net/http/server.go:2054 +0x44
k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1(0xc0013f9920, 0xc000772100, 0x2636360, 0xc0001f6ed0, 0xc00107ef00)
	k8s.io/apiserver.1-rc.0/pkg/server/filters/timeout.go:111 +0xb8
created by k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP
	k8s.io/apiserver.1-rc.0/pkg/server/filters/timeout.go:97 +0x1cc
```

result of result.Status at https://github.com/openshift/cluster-openshift-controller-manager-operator/blob/master/vendor/k8s.io/apiserver/plugin/pkg/authorizer/webhook/webhook.go#L208 is nil based on the panic trace. Most likely webhook.WithExponentialBackoff returns non-nil error which is not handled. Making result empty.

Fixed by https://github.com/kubernetes/kubernetes/pull/97820. Will get resolved during the rebase to 1.21. Worth backporting to older 4.X versions.

Comment 8 errata-xmlrpc 2021-07-27 22:54:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438


Note You need to log in before you can comment on or make changes to this bug.