+++ This bug was initially created as a clone of Bug #1886600 +++ https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-azure-4.5/1314281190638751744 https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_ovn-kubernetes/227/pull-ci-openshift-ovn-kubernetes-release-4.5-e2e-gcp-ovn-upgrade/1314289613564547072 panic: assignment to entry in nil map goroutine 80 [running]: k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) /go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:55 +0x105 panic(0x163d780, 0x1a61ab0) /usr/local/go/src/runtime/panic.go:679 +0x1b2 k8s.io/client-go/tools/leaderelection/resourcelock.(*ConfigMapLock).Update(0xc0000f6000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/client-go/tools/leaderelection/resourcelock/configmaplock.go:90 +0x129 k8s.io/client-go/tools/leaderelection.(*LeaderElector).release(0xc000f56b40, 0x6fc23ac00) /go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:306 +0xc9 k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew(0xc000f56b40, 0x1abc1a0, 0xc0004a4180) /go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:294 +0x166 k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run(0xc000f56b40, 0x1abc1a0, 0xc0004a4140) /go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:208 +0x15d k8s.io/client-go/tools/leaderelection.RunOrDie(0x1abc1a0, 0xc00030de00, 0x1acbcc0, 0xc0000f6000, 0x14f46b0400, 0xa7a358200, 0x6fc23ac00, 0xc001060f30, 0xc0010baa10, 0x0, ...) /go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:221 +0x96 github.com/openshift/cluster-version-operator/pkg/start.(*Options).run.func4(0x1abc1a0, 0xc00030de00, 0xc0000f6000, 0xc000f3ab40, 0xc000099d80, 0x1abc1a0, 0xc00030dd80, 0xc00094b500, 0xc0010ba9d0) /go/src/github.com/openshift/cluster-version-operator/pkg/start/start.go:179 +0x1b2 created by github.com/openshift/cluster-version-operator/pkg/start.(*Options).run /go/src/github.com/openshift/cluster-version-operator/pkg/start/start.go:177 +0x3f9 --- Additional comment from Scott Dodson on 2020-10-09 00:09:02 UTC --- Both of these are in 4.5 where we recently merged https://github.com/openshift/cluster-version-operator/pull/446 which is relevant to leader election, this is probably not a 4.6 blocker but we'll evaluate deeper tomorrow, leaving target release unassigned for now. --- Additional comment from W. Trevor King on 2020-10-09 04:19:58 UTC --- Looks like we need an upstream client-go fix backported. I'll try to figure out what our process is for that... --- Additional comment from W. Trevor King on 2020-10-09 05:51:19 UTC --- I've filed an upstream PR requesting a backport to 1.17. --- Additional comment from Scott Dodson on 2020-10-09 13:51:13 UTC --- This is 4.5 specific but the bot will revert my changes if i set target release to 4.5 directly, so setting it 4.7 now to get off the blocker list. --- Additional comment from W. Trevor King on 2020-10-09 18:51:56 UTC --- 4.6+ use 1.19 clients which have this patched, so setting this modified to unblock a new-in-4.5 patch.
Here's the patch landing in 4.6+: https://github.com/openshift/cluster-version-operator/commit/fb84815ff1eb5ff21378965acc497b526b839f69#diff-c554e6c97e8d61bb9449784b8630b2abR91
I'm verifying it on 4.6.0-0.nightly-2020-10-10-041109. Steps of verification are as below: 1. Fresh install an Azure cluster -- passed 2. Set up an OVN cluster on GCP with 4.5.14, and then do an upgrade to 4.6.0-0.nightly-2020-10-10-041109 - passed There's no panic found in CVO, hence moving it to verified state.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4196