Bug 1877919 - panic in multus-admission-controller
Summary: panic in multus-admission-controller
Keywords:
Status: VERIFIED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.6
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.7.0
Assignee: Douglas Smith
QA Contact: Weibin Liang
URL:
Whiteboard:
: 1877208 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-09-10 18:48 UTC by David Eads
Modified: 2020-09-18 19:37 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github openshift multus-admission-controller pull 27 None closed bug 1877919: tolerate missed watch event for delete 2020-09-22 14:50:33 UTC

Description David Eads 2020-09-10 18:48:22 UTC
pods/openshift-multus_multus-admission-controller-dlnrf_multus-admission-controller_previous.log.gz:E0910 10:19:45.911352       1 runtime.go:73] Observed a panic: &runtime.TypeAssertionError{_interface:(*runtime._type)(nil), concrete:(*runtime._type)(0x1175e80), asserted:(*runtime._type)(0x1225320), missingMethod:"GetAnnotations"} (interface conversion: cache.DeletedFinalStateUnknown is not v1.Object: missing method GetAnnotations)

E0910 10:19:45.911352       1 runtime.go:73] Observed a panic: &runtime.TypeAssertionError{_interface:(*runtime._type)(nil), concrete:(*runtime._type)(0x1175e80), asserted:(*runtime._type)(0x1225320), missingMethod:"GetAnnotations"} (interface conversion: cache.DeletedFinalStateUnknown is not v1.Object: missing method GetAnnotations)
goroutine 85 [running]:
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/apimachinery/pkg/util/runtime.logPanic(0x1131d60, 0xc000593590)
	/go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:69 +0x7b
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:51 +0x82
panic(0x1131d60, 0xc000593590)
	/usr/lib/golang/src/runtime/panic.go:969 +0x166
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/controller.newResourceController.func3(0x1175e80, 0xc000111bc0)
	/go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/pkg/controller/controller.go:155 +0x57
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnDelete(...)
	/go/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/gopath/src/github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/client-go/tools/cache/controller.go:209
github.com/k8snetworkplumbingwg/net-attach-def-admission-controller/vendor/k8s.io/client-go/tools/cache.(*processorListener).run.func1.1(0x0, 0x0, 0x0)

Comment 2 Douglas Smith 2020-09-11 13:52:50 UTC
*** Bug 1877208 has been marked as a duplicate of this bug. ***

Comment 5 Maru Newby 2020-09-17 15:07:58 UTC
This is easy to fix, so why not fix it ASAP for 4.6? Leaving unexpected panics latent in the code inures engineers to seeing these errors in their CI results, which can only hurt us.

Comment 6 Douglas Smith 2020-09-17 15:31:29 UTC
We have the proposed fix merged in https://bugzilla.redhat.com/show_bug.cgi?id=1877919 and while this makes sense to me, I'm unsure how to diagnose why this hasn't fixed in according to the link from Weibin.

My current step is to check that the build of the multus-admission-controller indeed has this code, which I'm in the process of doing.

Agreed that it would be much better to have in 4.6.

Comment 7 Douglas Smith 2020-09-17 18:03:37 UTC
Correction: merged in https://github.com/openshift/multus-admission-controller/pull/27

Comment 10 Weibin Liang 2020-09-18 19:37:58 UTC
Do not see any more multus panic log shown in v4.6 job


Note You need to log in before you can comment on or make changes to this bug.