Bug 1886939

Summary: panic: assignment to entry in nil map
Product: OpenShift Container Platform Reporter: W. Trevor King <wking>
Component: Cluster Version OperatorAssignee: W. Trevor King <wking>
Status: CLOSED ERRATA QA Contact: Yang Yang <yanyang>
Severity: high Docs Contact:
Priority: high    
Version: 4.5CC: aos-bugs, aos-team-ota, bgilbert, ivn13452, jialiu, jokerman, tdigitalguru, wking, yanyang
Target Milestone: ---   
Target Release: 4.5.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: 1886935 Environment:
Undiagnosed panic detected in pod
Last Closed: 2020-10-19 14:55:01 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1886935    
Bug Blocks:    

Description W. Trevor King 2020-10-09 18:58:11 UTC 
+++ This bug was initially created as a clone of Bug #1886935 +++

+++ This bug was initially created as a clone of Bug #1886600 +++

https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-azure-4.5/1314281190638751744
https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_ovn-kubernetes/227/pull-ci-openshift-ovn-kubernetes-release-4.5-e2e-gcp-ovn-upgrade/1314289613564547072

panic: assignment to entry in nil map

goroutine 80 [running]:
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:55 +0x105
panic(0x163d780, 0x1a61ab0)
	/usr/local/go/src/runtime/panic.go:679 +0x1b2
k8s.io/client-go/tools/leaderelection/resourcelock.(*ConfigMapLock).Update(0xc0000f6000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/client-go/tools/leaderelection/resourcelock/configmaplock.go:90 +0x129
k8s.io/client-go/tools/leaderelection.(*LeaderElector).release(0xc000f56b40, 0x6fc23ac00)
	/go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:306 +0xc9
k8s.io/client-go/tools/leaderelection.(*LeaderElector).renew(0xc000f56b40, 0x1abc1a0, 0xc0004a4180)
	/go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:294 +0x166
k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run(0xc000f56b40, 0x1abc1a0, 0xc0004a4140)
	/go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:208 +0x15d
k8s.io/client-go/tools/leaderelection.RunOrDie(0x1abc1a0, 0xc00030de00, 0x1acbcc0, 0xc0000f6000, 0x14f46b0400, 0xa7a358200, 0x6fc23ac00, 0xc001060f30, 0xc0010baa10, 0x0, ...)
	/go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:221 +0x96
github.com/openshift/cluster-version-operator/pkg/start.(*Options).run.func4(0x1abc1a0, 0xc00030de00, 0xc0000f6000, 0xc000f3ab40, 0xc000099d80, 0x1abc1a0, 0xc00030dd80, 0xc00094b500, 0xc0010ba9d0)
	/go/src/github.com/openshift/cluster-version-operator/pkg/start/start.go:179 +0x1b2
created by github.com/openshift/cluster-version-operator/pkg/start.(*Options).run
	/go/src/github.com/openshift/cluster-version-operator/pkg/start/start.go:177 +0x3f9

--- Additional comment from Scott Dodson on 2020-10-09 00:09:02 UTC ---

Both of these are in 4.5 where we recently merged https://github.com/openshift/cluster-version-operator/pull/446 which is relevant to leader election, this is probably not a 4.6 blocker but we'll evaluate deeper tomorrow, leaving target release unassigned for now.

--- Additional comment from W. Trevor King on 2020-10-09 04:19:58 UTC ---

Looks like we need an upstream client-go fix backported.  I'll try to figure out what our process is for that...

--- Additional comment from W. Trevor King on 2020-10-09 05:51:19 UTC ---

I've filed an upstream PR requesting a backport to 1.17.

--- Additional comment from Scott Dodson on 2020-10-09 13:51:13 UTC ---

This is 4.5 specific but the bot will revert my changes if i set target release to 4.5 directly, so setting it 4.7 now to get off the blocker list.

--- Additional comment from W. Trevor King on 2020-10-09 18:51:56 UTC ---

4.6+ use 1.19 clients which have this patched, so setting this modified to unblock a new-in-4.5 patch.

--- Additional comment from W. Trevor King on 2020-10-09 18:57:47 UTC ---

Here's the patch landing in 4.6+: https://github.com/openshift/cluster-version-operator/commit/fb84815ff1eb5ff21378965acc497b526b839f69#diff-c554e6c97e8d61bb9449784b8630b2abR91
Comment 3 Yang Yang 2020-10-13 06:11:58 UTC 
Verified on below use cases with 4.5.0-0.nightly-2020-10-10-030038

1. Azure cluster installation on 4.5.0-0.nightly-2020-10-10-030038 - passed
2. GCP OVN cluster upgrades to 4.5.0-0.nightly-2020-10-10-030038 - passed

There's no panic found in CVO, and hence moving it to verified state.
Comment 5 errata-xmlrpc 2020-10-19 14:55:01 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.5.15 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4228
Comment 6 Allenwood 2022-12-14 08:17:50 UTC Comment hidden (spam)
Comment 7 TDigitalguru 2023-05-04 16:18:34 UTC 
Thank you for the information - I have this guide - https://www.tdigitalguru.com/career-in-digital-marketing/