Bug 1877919 - panic in multus-admission-controller
Summary: panic in multus-admission-controller
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.6
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.7.0
Assignee: Douglas Smith
QA Contact: Weibin Liang
URL:
Whiteboard:
: 1877208 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-09-10 18:48 UTC by David Eads
Modified: 2021-02-24 15:18 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
(this couldn't be replicated)
Clone Of:
Environment:
Last Closed: 2021-02-24 15:17:43 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift multus-admission-controller pull 27 0 None closed bug 1877919: tolerate missed watch event for delete 2021-02-17 21:57:17 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:18:17 UTC

Description David Eads 2020-09-10 18:48:22 UTC
pods/openshift-multus_multus-admission-controller-dlnrf_multus-admission-controller_previous.log.gz:E0910 10:19:45.911352       1 runtime.go:73] Observed a panic: &runtime.TypeAssertionError{_interface:(*runtime._type)(nil), concrete:(*runtime._type)(0x1175e80), asserted:(*runtime._type)(0x1225320), missingMethod:"GetAnnotations"} (interface conversion: cache.DeletedFinalStateUnknown is not v1.Object: missing method GetAnnotations)

E0910 10:19:45.911352       1 runtime.go:73] Observed a panic: &runtime.TypeAssertionError{_interface:(*runtime._type)(nil), concrete:(*runtime._type)(0x1175e80), asserted:(*runtime._type)(0x1225320), missingMethod:"GetAnnotations"} (interface conversion: cache.DeletedFinalStateUnknown is not v1.Object: missing method GetAnnotations)
goroutine 85 [running]:
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/apimachinery/pkg/util/runtime.logPanic(0x1131d60, 0xc000593590)
	/go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:69 +0x7b
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:51 +0x82
panic(0x1131d60, 0xc000593590)
	/usr/lib/golang/src/runtime/panic.go:969 +0x166
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/controller.newResourceController.func3(0x1175e80, 0xc000111bc0)
	/go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/controller/controller.go:155 +0x57
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnDelete(...)
	/go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/client-go/tools/cache/controller.go:209
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/client-go/tools/cache.(*processorListener).run.func1.1(0x0, 0x0, 0x0)

Comment 2 Douglas Smith 2020-09-11 13:52:50 UTC
*** Bug 1877208 has been marked as a duplicate of this bug. ***

Comment 5 Maru Newby 2020-09-17 15:07:58 UTC
This is easy to fix, so why not fix it ASAP for 4.6? Leaving unexpected panics latent in the code inures engineers to seeing these errors in their CI results, which can only hurt us.

Comment 6 Douglas Smith 2020-09-17 15:31:29 UTC
We have the proposed fix merged in https://bugzilla.redhat.com/show_bug.cgi?id=1877919 and while this makes sense to me, I'm unsure how to diagnose why this hasn't fixed in according to the link from Weibin.

My current step is to check that the build of the multus-admission-controller indeed has this code, which I'm in the process of doing.

Agreed that it would be much better to have in 4.6.

Comment 7 Douglas Smith 2020-09-17 18:03:37 UTC
Correction: merged in https://github.com/openshift/multus-admission-controller/pull/27

Comment 10 Weibin Liang 2020-09-18 19:37:58 UTC
Do not see any more multus panic log shown in v4.6 job

Comment 13 errata-xmlrpc 2021-02-24 15:17:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.