Bug 1781293
| Summary: | KubeVirt kind hangs in Deleting state on cleanup script | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Container Native Virtualization (CNV) | Reporter: | Irina Gulina <igulina> | ||||
| Component: | Installation | Assignee: | Simone Tiraboschi <stirabos> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Tareq Alayan <talayan> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | medium | ||||||
| Version: | 2.2.0 | CC: | cnv-qe-bugs, fdeutsch, ncredi, rmohr, stirabos, talayan | ||||
| Target Milestone: | --- | Keywords: | TestOnly | ||||
| Target Release: | 2.3.0 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | hyperconverged-cluster-operator-container-v2.3.0-47 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2020-05-04 19:10:37 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | 1712429 | ||||||
| Bug Blocks: | |||||||
| Attachments: |
|
||||||
Created attachment 1643380 [details] logs for comment #0 The cleanup script is not released to customers, thus a low priority. @Roman, Is this fixed already? what is the cleanup script? It's a cleanup script by hco. But this bug is specifically about hanging when the kubevirt CR is getting deletde. And IIUIC then you were working on this in upstream, right? If we talk about this script https://github.com/kubevirt/hyperconverged-cluster-operator/blob/master/hack/clean.sh, then https://github.com/kubevirt/kubevirt/pull/3006 should solve it, but it is not yet merged. Ack. Changing this bug to then be test only once the other bug is resolved We refer to https://github.com/kubevirt/hyperconverged-cluster-operator/blob/master/deploy/cleanup_marketplace.sh It still doesn't work, since HCO resource instance name is wrong It should be "kubevirt-hyperconverged" and not "hyperconverged-cluster" according line #12 of CSV file. The bug should have been fixed with https://github.com/kubevirt/hyperconverged-cluster-operator/commit/c0852e0771b0dce47eaab43d518096a2ac6d461b#diff-e0269f18a4a80faa4062b32f9f32c0f0 Moving back to ON_QA curl -k https://raw.githubusercontent.com/kubevirt/hyperconverged-cluster-operator/master/deploy/cleanup_marketplace.sh | TARGET=openshift-cnv APP_REGISTRY=rh-verified-operators HCO_VERSION=2.3.0 bash -x % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 694 100 694 0 0 2238 0 --:--:-- --:--:-- --:--:-- 2238 + MARKETPLACE=openshift-marketplace + TARGET=openshift-cnv + APP_REGISTRY=rh-verified-operators + HCO_VERSION=2.3.0 + oc delete csc hco-catalogsource-config -n openshift-marketplace catalogsourceconfig.operators.coreos.com "hco-catalogsource-config" deleted + oc delete catalogsource rh-verified-operators -n openshift-marketplace catalogsource.operators.coreos.com "rh-verified-operators" deleted + oc delete operatorsource rh-verified-operators -n openshift-marketplace operatorsource.operators.coreos.com "rh-verified-operators" deleted + oc delete hco kubevirt-hyperconverged -n openshift-cnv hyperconverged.hco.kubevirt.io "kubevirt-hyperconverged" deleted + sleep 10 + oc delete sub hco-operatorhub -n openshift-cnv Error from server (NotFound): subscriptions.operators.coreos.com "hco-operatorhub" not found + oc delete csv kubevirt-hyperconverged-operator.v2.3.0 -n openshift-cnv clusterserviceversion.operators.coreos.com "kubevirt-hyperconverged-operator.v2.3.0" deleted + oc delete operatorgroup openshift-cnv-group -n openshift-cnv operatorgroup.operators.coreos.com "openshift-cnv-group" deleted ++ oc get secret -n openshift-marketplace ++ grep quay-registry ++ awk '{print $1}' + oc delete secret quay-registry-redhat-operators-stage quay-registry-rh-verified-operators -n openshift-marketplace secret "quay-registry-redhat-operators-stage" deleted secret "quay-registry-rh-verified-operators" deleted [cloud-user@ocp-psi-executor ~]$ watch oc get pods -n openshift-cnv [cloud-user@ocp-psi-executor ~]$ oc get pods -n openshift-cnv No resources found in openshift-cnv namespace. [cloud-user@ocp-psi-executor ~]$ oc get csc -n openshift-marketplace No resources found in openshift-marketplace namespace. [cloud-user@ocp-psi-executor ~]$ oc get catalogsource -n openshift-marketplace No resources found in openshift-marketplace namespace. [cloud-user@ocp-psi-executor ~]$ oc get operatorsource -n openshift-marketplace No resources found in openshift-marketplace namespace. [cloud-user@ocp-psi-executor ~]$ oc get hco -n openshift-marketplace No resources found in openshift-marketplace namespace. [cloud-user@ocp-psi-executor ~]$ oc get all -n openshift-marketplace NAME READY STATUS RESTARTS AGE pod/marketplace-operator-7864756bd9-bn4hz 1/1 Running 0 23h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/marketplace-operator-metrics ClusterIP 172.30.158.12 <none> 8383/TCP 23h NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/marketplace-operator 1/1 1 1 23h NAME DESIRED CURRENT READY AGE replicaset.apps/marketplace-operator-7864756bd9 1 1 1 23h [cloud-user@ocp-psi-executor ~]$ oc get secret -n openshift-marketplace NAME TYPE DATA AGE builder-dockercfg-qfh45 kubernetes.io/dockercfg 1 23h builder-token-qzllz kubernetes.io/service-account-token 4 23h builder-token-tqghx kubernetes.io/service-account-token 4 23h default-dockercfg-t52br kubernetes.io/dockercfg 1 23h default-token-qqwr4 kubernetes.io/service-account-token 4 23h default-token-vs2q4 kubernetes.io/service-account-token 4 23h deployer-dockercfg-9gkhq kubernetes.io/dockercfg 1 23h deployer-token-jlfjv kubernetes.io/service-account-token 4 23h deployer-token-qwn86 kubernetes.io/service-account-token 4 23h marketplace-operator-dockercfg-fzjq4 kubernetes.io/dockercfg 1 23h marketplace-operator-token-nbm7b kubernetes.io/service-account-token 4 23h marketplace-operator-token-zv5th kubernetes.io/service-account-token 4 23h [cloud-user@ocp-psi-executor ~]$ oc get all -n openshift-cnv No resources found in openshift-cnv namespace. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2020:2011 |
Description of problem: KubeVirt kind is hanging in Deleting state on cleanup script. Version-Release number of selected component (if applicable): hco:v2.2.0-9 How reproducible: always Steps to Reproduce: install cnv 2.2 with the marketplace script, run the cleanup script Actual results: see the attachment, notice that lines with a timestamp: >>[cnv-qe-jenkins@cnv-executor-alexxa-43 ~]$ date >> Mon Dec 9 16:26:06 UTC 2019 >>[cnv-qe-jenkins@cnv-executor-alexxa-43 ~]$ watch oc get pods -n openshift-cnv >>[cnv-qe-jenkins@cnv-executor-alexxa-43 ~]$ date >> Mon Dec 9 17:15:43 UTC 2019 Expected results: no resources left