Bug 1917904

This BZ will fix the following panic message that has already surfaced in CI (https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-openshift-tests-private-release-4.7-sanity/1351922587822723072)


E0125 15:16:38.289815       1 runtime.go:76] Observed a panic: runtime error: invalid memory address or nil pointer dereference
goroutine 224651 [running]:
k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1.1(0xc007e935c0)
	k8s.io/apiserver.1/pkg/server/filters/timeout.go:106 +0x113
panic(0x2a0f120, 0x473beb0)
	runtime/panic.go:969 +0x1b9
k8s.io/apiserver/plugin/pkg/authorizer/webhook.(*WebhookAuthorizer).Authorize(0xc00052bb60, 0x338dda0, 0xc0079533e0, 0x33b4de0, 0xc0087bdd60, 0x2, 0x0, 0x0, 0x0, 0x0)

1.20.3 is going to be released on 2021-02-17, according to https://github.com/kubernetes/sig-release/blob/master/releases/patch-releases.md#cadence

Today hit same as comment 1's must-gather logs in UPI_vSphere_https_proxy_fips on_ovn env, one (and only one) of the oauth-apiserver pod instances hit:
$ oc get clusterversion
version   4.7.0-0.nightly-2021-02-02-164630   True        False         30h     Cluster version is 4.7.0-0.nightly-2021-02-02-164630

$ oc logs --timestamps apiserver-86bf876bc-b25qn -n openshift-oauth-apiserver | tee logs/oauth_apiserver-86bf876bc-b25qn.log
2021-02-04T02:45:37.360652417Z I0204 02:45:37.360617       1 trace.go:205] Trace[487475631]: "Get" url:/apis/oauth.openshift.io/v1/oauthaccesstokens/sha256~XcG9_XkhRlK5R6UgLJU5sVMVGEtntxZ4fa4GiPwm8A8,user-agent:kube-apiserver/v1.20.0+3d0efee (linux/amd64) kubernetes/3d0efee,client:::1 (04-Feb-2021 02:44:45.052) (total time: 52308ms):
...
2021-02-04T02:45:42.096669829Z E0204 02:45:42.096601       1 runtime.go:76] Observed a panic: runtime error: invalid memory address or nil pointer dereference
2021-02-04T02:45:42.096669829Z goroutine 778814 [running]:
...
2021-02-04T02:45:42.096669829Z k8s.io/apiserver/plugin/pkg/authorizer/webhook.(*WebhookAuthorizer).Authorize(0xc000a154a0, 0x2720b40, 0xc00083ea50, 0x273fbc0, 0xc002814c80, 0x2, 0xc001bf65f0, 0xeb51ef, 0x2721200, 0xc001aec7c0)
...
2021-02-04T02:45:42.096669829Z k8s.io/apiserver/pkg/server/filters.WithPriorityAndFairness.func1.4()
2021-02-04T02:45:42.096669829Z  k8s.io/apiserver.1/pkg/server/filters/priority-and-fairness.go:127 +0x3c6
...
Extracted the rest of panic places:
2021-02-04T02:45:42.096743810Z E0204 02:45:42.096637       1 wrap.go:58] apiserver panic'd on GET /apis/user.openshift.io/v1/users/~
2021-02-04T02:45:42.096760340Z E0204 02:45:42.096729       1 runtime.go:76] Observed a panic: runtime error: invalid memory address or nil pointer dereference

2021-02-04T02:45:42.096829089Z E0204 02:45:42.096762       1 wrap.go:58] apiserver panic'd on GET /apis/user.openshift.io/v1/users/~
2021-02-04T02:45:42.096829089Z E0204 02:45:42.096762       1 runtime.go:76] Observed a panic: runtime error: invalid memory address or nil pointer dereference

2021-02-04T02:45:42.096870658Z E0204 02:45:42.096787       1 wrap.go:58] apiserver panic'd on GET /apis/user.openshift.io/v1/users/~
2021-02-04T02:45:42.097690246Z E0204 02:45:42.097368       1 runtime.go:76] Observed a panic: runtime error: invalid memory address or nil pointer dereference
Full log http://file.rdu.redhat.com/~xxia/bug/1917904/oauth_apiserver-86bf876bc-b25qn.log is uploaded.

lszaszki,

> 2021-02-04T02:45:42.096669829Z k8s.io/apiserver/plugin/pkg/authorizer/webhook.(*WebhookAuthorizer).Authorize(0xc000a154a0, 0x2720b40, 0xc00083ea50, 0x273fbc0, 0xc002814c80, 0x2, 0xc001bf65f0, 0xeb51ef, 0x2721200, 0xc001aec7c0)
...
2021-02-04T02:45:42.096669829Z k8s.io/apiserver/pkg/server/filters.WithPriorityAndFairness.func1.4()
2021-02-04T02:45:42.096669829Z  k8s.io/apiserver.1/pkg/server/filters/priority-and-fairness.go:127 +0x3c6

side note: should openshift-oauth-apiserver have p&f enabled?

The fix applies to both openshift-apiserver and oauth-apiserver

*** Bug 1917906 has been marked as a duplicate of this bug. ***

PRs are ready to be merged.

Using the verification approach in bug 1933599#c2 , check 4.7 jobs in last days since PR merged, did not find the previous encountered error in openshift-apiserver and oauth-apiserver now:
https://search.ci.openshift.org/?search=apiserver.*Observed+a+panic%3A+runtime+error%3A+invalid+memory+address+or+nil+pointer+dereference&maxAge=168h&context=1&type=junit&name=4%5C.7&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job
So moving to VERIFIED.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.5 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:1005