Bug 1875353

Summary: Undiagnosed panic detected in amazon-eks-pod-identity-webhook pod
Product: OpenShift Container Platform Reporter: Michal Fojtik <mfojtik>
Component: Cloud Credential OperatorAssignee: Seth Jennings <sjenning>
Status: CLOSED ERRATA QA Contact: wang lin <lwan>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 4.6CC: lwan, sjenning
Target Milestone: ---Flags: lwan: needinfo-
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 16:37:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Michal Fojtik 2020-09-03 11:36:11 UTC
Description of problem:

E0902 18:40:00.201247       1 runtime.go:69] Observed a panic: &runtime.TypeAssertionError{_interface:(*runtime._type)(0x11545a0), concrete:(*runtime._type)(0x11c6960), asserted:(*runtime._type)(0x12c2160), missingMethod:""} (interface conversion: interface {} is cache.DeletedFinalStateUnknown, not *v1.ServiceAccount)
/go/src/github.com/aws/amazon-eks-pod-identity-webhook/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:76
/go/src/github.com/aws/amazon-eks-pod-identity-webhook/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:65
/go/src/github.com/aws/amazon-eks-pod-identity-webhook/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:51
/usr/lib/golang/src/runtime/panic.go:969
/usr/lib/golang/src/runtime/iface.go:260
/go/src/github.com/aws/amazon-eks-pod-identity-webhook/pkg/cache/cache.go:121
/go/src/github.com/aws/amazon-eks-pod-identity-webhook/vendor/k8s.io/client-go/tools/cache/controller.go:209
/go/src/github.com/aws/amazon-eks-pod-identity-webhook/vendor/k8s.io/client-go/tools/cache/controller.go:373
/go/src/github.com/aws/amazon-eks-pod-identity-webhook/vendor/k8s.io/client-go/tools/cache/delta_fifo.go:436
/go/src/github.com/aws/amazon-eks-pod-identity-webhook/vendor/k8s.io/client-go/tools/cache/controller.go:150
/go/src/github.com/aws/amazon-eks-pod-identity-webhook/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152
/go/src/github.com/aws/amazon-eks-pod-identity-webhook/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153
/go/src/github.com/aws/amazon-eks-pod-identity-webhook/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88
/go/src/github.com/aws/amazon-eks-pod-identity-webhook/vendor/k8s.io/client-go/tools/cache/controller.go:124
/usr/lib/golang/src/runtime/asm_amd64.s:1373

Detected in: https://prow.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-origin-installer-e2e-aws-disruptive-4.6/1301211648454823936

Fix in: https://github.com/aws/amazon-eks-pod-identity-webhook/pull/70


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Michal Fojtik 2020-09-03 11:36:42 UTC
Severity is urgent because unhandled panics now fail the test jobs.

Comment 2 Seth Jennings 2020-09-03 13:02:22 UTC
upstream PR https://github.com/aws/amazon-eks-pod-identity-webhook/pull/70

Comment 5 wang lin 2020-09-10 13:35:13 UTC
Hi Michal, sorry for bother you. Does this one need a QE verify? I can't see this issue in the latest nightly build. Does this PR has fixed the issue and no longer block your work? if so, I will move the status to verified.

Comment 6 wang lin 2020-09-30 04:34:30 UTC
Sorry for tracking it late
Has fixed, the issue has gone in the next e2e test: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-origin-installer-e2e-aws-disruptive-4.6/1302661504373362688

Comment 8 errata-xmlrpc 2020-10-27 16:37:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196