Bug 1982693
| Summary: | olm Operator is in CrashLoopBackOff state with error "couldn't cleanup cross-namespace ownerreferences" | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Kevin Rizza <krizza> |
| Component: | OLM | Assignee: | Kevin Rizza <krizza> |
| OLM sub component: | OLM | QA Contact: | Jian Zhang <jiazha> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | urgent | ||
| Priority: | urgent | CC: | akashem, davegord, dsover, jiazha, sreber |
| Version: | 4.7 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.7.z | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 1982252 | Environment: | |
| Last Closed: | 2021-08-03 17:56:24 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1982252 | ||
| Bug Blocks: | |||
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.7.22 bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:2903 |
[cloud-user@preserve-olm-env jian]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.7.0-0.nightly-2021-07-24-034734 True False 4m17s Cluster version is 4.7.0-0.nightly-2021-07-24-034734 [cloud-user@preserve-olm-env jian]$ oc -n openshift-operator-lifecycle-manager exec deploy/catalog-operator -- olm --version OLM version: 0.17.0 git commit: dcbab9aff049c39d3a8a4b544d8829b7ce93d8f8 This payload contains the fixed PR. 1, Install some cluster-scoped operators. [cloud-user@preserve-olm-env jian]$ oc get sub -A NAMESPACE NAME PACKAGE SOURCE CHANNEL openshift-operators-redhat elasticsearch-operator elasticsearch-operator redhat-operators stable openshift-operators argocd-operator argocd-operator community-operators alpha openshift-operators etcd etcd community-operators clusterwide-alpha openshift-operators jaeger jaeger community-operators stable openshift-operators servicemeshoperator servicemeshoperator redhat-operators stable [cloud-user@preserve-olm-env jian]$ oc get csv -n openshift-operators NAME DISPLAY VERSION REPLACES PHASE argocd-operator.v0.0.15 Argo CD 0.0.15 argocd-operator.v0.0.14 Succeeded elasticsearch-operator.5.1.0-96 OpenShift Elasticsearch Operator 5.1.0-96 Succeeded etcdoperator.v0.9.4-clusterwide etcd 0.9.4-clusterwide etcdoperator.v0.9.2-clusterwide Succeeded jaeger-operator.v1.24.0 Community Jaeger Operator 1.24.0 jaeger-operator.v1.21.3 Succeeded servicemeshoperator.v2.0.6.2 Red Hat OpenShift Service Mesh 2.0.6-2 servicemeshoperator.v2.0.6.1 Succeeded [cloud-user@preserve-olm-env jian]$ oc get csv -A|wc -l 322 [cloud-user@preserve-olm-env jian]$ oc get ns |wc -l 65 2, Create 300 namespaces [cloud-user@preserve-olm-env jian]$ for l in {1..300}; do oc adm new-project "test$l";sleep 1; done; ... 3, Check the CSV numbers and the OLM pods. [cloud-user@preserve-olm-env jian]$ oc get csv -A|wc -l 1822 [cloud-user@preserve-olm-env jian]$ oc get pods -n openshift-operator-lifecycle-manager NAME READY STATUS RESTARTS AGE catalog-operator-f966bdcbf-gtjr6 1/1 Running 0 64m olm-operator-5696d565d7-jvnjx 1/1 Running 0 64m packageserver-7d67fd99cb-b4ds7 1/1 Running 0 58m packageserver-7d67fd99cb-j8mw4 1/1 Running 0 58m 4, OLM pods works well, install some new operators. Looks good. [cloud-user@preserve-olm-env jian]$ oc get csv -n openshift-operators NAME DISPLAY VERSION REPLACES PHASE argocd-operator.v0.0.15 Argo CD 0.0.15 argocd-operator.v0.0.14 Succeeded elasticsearch-operator.5.1.0-96 OpenShift Elasticsearch Operator 5.1.0-96 Succeeded etcdoperator.v0.9.4-clusterwide etcd 0.9.4-clusterwide etcdoperator.v0.9.2-clusterwide Succeeded jaeger-operator.v1.24.0 Community Jaeger Operator 1.24.0 jaeger-operator.v1.21.3 Succeeded nfd.4.7.0-202106090743 Node Feature Discovery 4.7.0-202106090743 Installing servicemeshoperator.v2.0.6.2 Red Hat OpenShift Service Mesh 2.0.6-2 servicemeshoperator.v2.0.6.1 Succeeded [cloud-user@preserve-olm-env jian]$ oc get csv -A |wc -l 2187 5, Remove all cluster-scoped operators, [cloud-user@preserve-olm-env jian]$ oc get csv -n openshift-operators No resources found in openshift-operators namespace. [cloud-user@preserve-olm-env jian]$ oc get csv -A |wc -l 3 [cloud-user@preserve-olm-env jian]$ oc get pods -n openshift-operator-lifecycle-manager NAME READY STATUS RESTARTS AGE catalog-operator-f966bdcbf-gtjr6 1/1 Running 0 72m olm-operator-5696d565d7-jvnjx 1/1 Running 0 72m packageserver-7d67fd99cb-b4ds7 1/1 Running 0 65m packageserver-7d67fd99cb-j8mw4 1/1 Running 0 65m No OLM pods crased, looks good, verify it.