Bug 2027824 - ovnkube-master CrashLoopBackoff: panic: Expected slice or struct but got string
Summary: ovnkube-master CrashLoopBackoff: panic: Expected slice or struct but got string
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.10
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.10.0
Assignee: Surya Seetharaman
QA Contact: Ross Brattain
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-11-30 19:25 UTC by Devan Goodwin
Modified: 2022-03-10 16:31 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-03-10 16:31:00 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift ovn-kubernetes pull 851/commits/33a517d17764a748388dcb1da17d6bd67fa682dd 0 None None None 2021-12-01 17:54:01 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-10 16:31:22 UTC

Description Devan Goodwin 2021-11-30 19:25:04 UTC 
TRT identified a crash scenario that looks like a clear code error in ovn.

Prow job: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.10-e2e-azure-ovn-upgrade/1465486322222239744

Crash log: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.10-e2e-azure-ovn-upgrade/1465486322222239744/artifacts/e2e-azure-ovn-upgrade/gather-must-gather/artifacts/must-gather/namespaces/openshift-ovn-kubernetes/pods/ovnkube-master-dl7zg/ovnkube-master/ovnkube-master/logs/previous.log

Which shows:

2021-11-30T02:27:25.994211476Z E1130 02:27:25.994173       1 master.go:163] Failed to remove any legacy per node join switches
2021-11-30T02:27:25.994211476Z I1130 02:27:25.994191       1 master.go:197] Upgrading to Namespace Deny PortGroup OVN Topology
2021-11-30T02:27:26.148705248Z panic: Expected slice or struct but got string
2021-11-30T02:27:26.148705248Z 
2021-11-30T02:27:26.148705248Z goroutine 49 [running]:
2021-11-30T02:27:26.148705248Z github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/libovsdbops.onModels({0x17ea8a0, 2021-11-30T02:27:26.148910652Z 0x2c46a60}, 0xc0002ad2f8)
2021-11-30T02:27:26.148910652Z 	/go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/libovsdbops/model.go:247 +0x379
2021-11-30T02:27:26.148910652Z github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/libovsdbops.(*ModelClient).buildOps(0xc0002ad438, 0xc0002ad368, 0x0, {0xc0002ad508, 0x1, 0x50})
2021-11-30T02:27:26.148910652Z 	/go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/libovsdbops/model_client.go:231 +0x4dd
2021-11-30T02:27:26.148910652Z github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/libovsdbops.(*ModelClient).DeleteOps(0xc0002ad3b0, {0xc0002ad508, 2021-11-30T02:27:26.148982554Z 0x50, 0x19f1380})
2021-11-30T02:27:26.148982554Z 	/go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/libovsdbops/model_client.go:201 +0x45
2021-11-30T02:27:26.148982554Z github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn/libovsdbops.(*ModelClient).Delete(0xc0002ad438, {0xc0002ad508, 0x23, 0xc0002ad478})
2021-11-30T02:27:26.148982554Z 	/go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/libovsdbops/model_client.go:185 +0x25
2021-11-30T02:27:26.149011954Z github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn.(*Controller).cleanupDGP(0xc00012b900, 0x0)
2021-11-30T02:27:26.149011954Z 	/go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/gateway_cleanup.go:3462021-11-30T02:27:26.149037455Z  +0x176
2021-11-30T02:27:26.149037455Z github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn.(*Controller).upgradeOVNTopology(0xc000000005, 0x0)
2021-11-30T02:27:26.149037455Z 	2021-11-30T02:27:26.149063555Z /go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/master.go:202 +0x136
2021-11-30T02:27:26.149063555Z github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn.(*Controller).StartClusterMaster(0xc00012b9002021-11-30T02:27:26.149089256Z , {0x7ffe4df868c9, 0x23})
2021-11-30T02:27:26.149089256Z 	/go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/master.go:2742021-11-30T02:27:26.149113456Z  +0x3f4
2021-11-30T02:27:26.149113456Z github.com/ovn-org/ovn-kubernetes/go-controller/pkg/ovn.(*Controller).Start.func1({0xc000f3cbf8, 0xc000f3cbfa2021-11-30T02:27:26.149137857Z })
2021-11-30T02:27:26.149137857Z 	/go/src/github.com/openshift/ovn-kubernetes/go-controller/pkg/ovn/master.go:98 +0x11a
2021-11-30T02:27:26.149160057Z created by k8s.io/client-go/tools/leaderelection.(*LeaderElector).Run
2021-11-30T02:27:26.149183558Z 	/go/src/github.com/openshift/ovn-kubernetes/go-controller/vendor/k8s.io/client-go/tools/leaderelection/leaderelection.go:211 +0x154
Comment 3 Ross Brattain 2021-12-09 20:38:23 UTC 
No "panic: Expected slice or struct but got string" in prow or recent 4.10-e2e-azure-ovn-upgrade

Verified
Comment 6 errata-xmlrpc 2022-03-10 16:31:00 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056
Note You need to log in before you can comment on or make changes to this bug.