Bug 1879426

Summary: registry operator panics: interface conversion: cache.DeletedFinalStateUnknown is not v1.Object: missing method GetAnnotations
Product: OpenShift Container Platform Reporter: Oleg Bulatov <obulatov>
Component: Image RegistryAssignee: Oleg Bulatov <obulatov>
Status: CLOSED ERRATA QA Contact: Wenjing Zheng <wzheng>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.5CC: aos-bugs, bgilbert
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: type assertions were made twice for a variable, and the second time its result was not checked Consequence: the assertions were false for cache.DeletedFinalStateUnknown and the second one caused panic Fix: use checked type assertions Result: the operator does not panic
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 16:41:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1886603    

Description Oleg Bulatov 2020-09-16 09:37:06 UTC
Description of problem:

The registry operator panics:

E0916 05:10:49.599799       1 runtime.go:78] Observed a panic: &runtime.TypeAssertionError{_interface:(*runtime._type)(nil), concrete:(*runtime._type)(0x239e580), asserted:(*runtime._type)(0x255f620), missingMethod:"GetAnnotations"} (interface conversion: cache.DeletedFinalStateUnknown is not v1.Object: missing method GetAnnotations)
goroutine 350 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x22d3d60, 0xc0010b0390)
	/go/src/github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:74 +0xa3
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/src/github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:48 +0x82
panic(0x22d3d60, 0xc0010b0390)
	/usr/lib/golang/src/runtime/panic.go:969 +0x166
github.com/openshift/cluster-image-registry-operator/pkg/operator.(*Controller).handler.func3(0x239e580, 0xc000c58100)
	/go/src/github.com/openshift/cluster-image-registry-operator/pkg/operator/controller.go:387 +0xab
k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnDelete(...)
	/go/src/github.com/openshift/cluster-image-registry-operator/vendor/k8s.io/client-go/tools/cache/controller.go:241

https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-origin-installer-e2e-azure-4.6/1306090255216021504

https://storage.googleapis.com/origin-ci-test/logs/release-openshift-origin-installer-e2e-azure-4.6/1306090255216021504/artifacts/e2e-azure/pods/openshift-image-registry_cluster-image-registry-operator-845f685745-87mld_cluster-image-registry-operator_previous.log

Comment 4 Benjamin Gilbert 2020-10-08 21:03:35 UTC
Since this is targeted for 4.6, I gather we're going to continue seeing test failures for it in 4.5?  Accordingly, I've added "Undiagnosed panic detected in pod" to the environment list, which will unfortunately cause unrelated undiagnosed panics to drop off of build-watchers' radar.

Comment 5 Benjamin Gilbert 2020-10-08 22:48:32 UTC
Removing the test from the environment list again, since I misunderstood how Sippy correlates BZs: https://github.com/openshift/sippy/issues/128#issuecomment-705797307

Comment 7 errata-xmlrpc 2020-10-27 16:41:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196