Bug 1952684
| Summary: | cluster-etcd-operator: metrics controller panics on invalid response from client | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Sam Batschelet <sbatsche> |
| Component: | Etcd | Assignee: | Sam Batschelet <sbatsche> |
| Status: | CLOSED ERRATA | QA Contact: | ge liu <geliu> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 4.5 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.8.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | No Doc Update | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-07-27 23:03:04 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This problem have not be hit in ci log again. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2438 |
Description of problem: panic observed in CI [1],[2] ``` E0415 15:58:49.986605 1 runtime.go:78] Observed a panic: runtime.boundsError{x:0, y:0, signed:true, code:0x0} (runtime error: index out of range [0] with length 0) goroutine 252 [running]: k8s.io/apimachinery/pkg/util/runtime.logPanic(0x2527900, 0xc002a0c258) k8s.io/apimachinery.0-rc.0/pkg/util/runtime/runtime.go:74 +0x95 k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0xc002c91d58, 0x1, 0x1) k8s.io/apimachinery.0-rc.0/pkg/util/runtime/runtime.go:48 +0x86 panic(0x2527900, 0xc002a0c258) runtime/panic.go:965 +0x1b9 github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller.(*FSyncController).sync(0xc000795820, 0x2a477b8, 0xc000ea0000, 0x2a42390, 0xc0009f43c0, 0x0, 0x0) github.com/openshift/cluster-etcd-operator/pkg/operator/metriccontroller/fsync_controller.go:56 +0xecc github.com/openshift/library-go/pkg/controller/factory.(*baseController).reconcile(0xc000e877a0, 0x2a477b8, 0xc000ea0000, 0x2a42390, 0xc0009f43c0, 0x0, 0x0) github.com/openshift/library-go.0-20210414082648-6e767630a0dc/pkg/controller/factory/base_controller.go:204 +0x56 github.com/openshift/library-go/pkg/controller/factory.(*baseController).processNextWorkItem(0xc000e877a0, 0x2a477b8, 0xc000ea0000) github.com/openshift/library-go.0-20210414082648-6e767630a0dc/pkg/controller/factory/base_controller.go:258 +0x253 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker.func1(0x2a477b8, 0xc000ea0000) github.com/openshift/library-go.0-20210414082648-6e767630a0dc/pkg/controller/factory/base_controller.go:195 +0xb9 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1() k8s.io/apimachinery.0-rc.0/pkg/util/wait/wait.go:185 +0x37 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00107bef8) k8s.io/apimachinery.0-rc.0/pkg/util/wait/wait.go:155 +0x5f k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc002c91ef8, 0x29eef80, 0xc000a337d0, 0xc000ea0001, 0xc000cdacc0) k8s.io/apimachinery.0-rc.0/pkg/util/wait/wait.go:156 +0x9b k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00107bef8, 0x3b9aca00, 0x0, 0x1, 0xc000cdacc0) k8s.io/apimachinery.0-rc.0/pkg/util/wait/wait.go:133 +0x98 k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext(0x2a477b8, 0xc000ea0000, 0xc00107bf58, 0x3b9aca00, 0x0, 0x1) k8s.io/apimachinery.0-rc.0/pkg/util/wait/wait.go:185 +0xa6 k8s.io/apimachinery/pkg/util/wait.UntilWithContext(...) k8s.io/apimachinery.0-rc.0/pkg/util/wait/wait.go:99 github.com/openshift/library-go/pkg/controller/factory.(*baseController).runWorker(0xc000e877a0, 0x2a477b8, 0xc000ea0000) github.com/openshift/library-go.0-20210414082648-6e767630a0dc/pkg/controller/factory/base_controller.go:186 +0x74 github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run.func2(0xc000e877a0, 0xc000056d50, 0x2a477b8, 0xc000ea0000) github.com/openshift/library-go.0-20210414082648-6e767630a0dc/pkg/controller/factory/base_controller.go:116 +0x75 created by github.com/openshift/library-go/pkg/controller/factory.(*baseController).Run github.com/openshift/library-go.0-20210414082648-6e767630a0dc/pkg/controller/factory/base_controller.go:111 +0x2f6 I0415 15:58:49.987493 1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"9c5c2e31-fbd5-421f-a288-247e6571b4c2", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'Openshift-Cluster-Etcd-OperatorPanic' Panic observed: runtime error: index out of range [0] with length 0 ``` [1] https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_clus[…]tcd-operator-7897fff8c9-njwzl_etcd-operator.log [2] https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_cluster-net[…]rk-operator-master-e2e-gcp-ovn-upgrade/1382687306791522304 Version-Release number of selected component (if applicable): How reproducible: unknown Steps to Reproduce: 1. 2. 3. Actual results: panic index out of range [0] with length 0 Expected results: response length verified greater than 0 before accessing first index of array. Additional info: