Fedora Account System
Red Hat Associate
Red Hat Customer
*Note*: this is a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1904830 that was resolved in 2.5.2 and encountered again in 2.5.3. Description of problem: Upgrade from 2.5.2 to 2.5.3 fails due to KubevirtCommonTemplatesBundle CR not updated with the new observedVersion. Version-Release number of selected component (if applicable): HCO bundle: rh-osbs/container-native-virtualization-hco-bundle-registry:v2.5.3-7 SSP image: https://access.redhat.com/containers/#/registry.access.redhat.com/container-native-virtualization/kubevirt-ssp-operator/images/v2.5.2-5 How reproducible: 100% Steps to Reproduce: 1. deploy CNV from production 2. upgrade to 2.5.3 from OSBS 3. Actual results: CSV is stuck on "Installing" state due to the mentioned issue. Expected results: The upgrade should be completed successfully and the CSV should be in "Succeeded" phase. Additional info: KubevirtCommonTemplatesBundle CR yaml: apiVersion: ssp.kubevirt.io/v1 kind: KubevirtCommonTemplatesBundle metadata: selfLink: >- /apis/ssp.kubevirt.io/v1/namespaces/openshift/kubevirtcommontemplatesbundles/common-templates-kubevirt-hyperconverged resourceVersion: '348718027' name: common-templates-kubevirt-hyperconverged uid: bf89cccb-bdc6-4393-8c18-303364ab82dd creationTimestamp: '2020-07-30T14:46:43Z' generation: 1 managedFields: - apiVersion: ssp.kubevirt.io/v1 fieldsType: FieldsV1 fieldsV1: 'f:metadata': 'f:labels': .: {} 'f:app': {} 'f:spec': {} manager: hyperconverged-cluster-operator operation: Update time: '2020-07-30T14:46:43Z' - apiVersion: ssp.kubevirt.io/v1 fieldsType: FieldsV1 fieldsV1: 'f:status': 'f:observedVersion': {} 'f:operatorVersion': {} 'f:targetVersion': {} manager: OpenAPI-Generator operation: Update time: '2020-12-17T21:00:30Z' - apiVersion: ssp.kubevirt.io/v1 fieldsType: FieldsV1 fieldsV1: 'f:status': .: {} 'f:conditions': {} manager: ansible-operator operation: Update time: '2020-12-22T08:10:48Z' namespace: openshift labels: app: kubevirt-hyperconverged spec: {} status: conditions: - lastTransitionTime: '2020-12-17T21:00:31Z' message: >- Templates progressing (deployed templates: 148, desired deployed templated: 58). reason: progressing status: 'False' type: Progressing - lastTransitionTime: '2020-12-17T21:00:31Z' message: >- Common templates available (deployed templates: 148, desired deployed templated: 58). reason: available status: 'True' type: Available - lastTransitionTime: '2020-12-17T21:00:31Z' message: >- Templates degraded (deployed templates: 148, desired deployed templated: 58). reason: degraded status: 'False' type: Degraded - lastTransitionTime: '2020-12-22T08:10:12Z' message: Running reconciliation reason: Running status: 'False' type: Running - lastTransitionTime: '2020-12-22T08:10:48Z' message: >- An unhandled exception occurred while running the lookup plugin 'k8s'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Failed to find exact match for kubevirt.io/v1.KubevirtCommonTemplatesBundle by [kind, name, singularName, shortNames] reason: Failed status: 'True' type: Failure observedVersion: v2.5.2 operatorVersion: v2.5.3 targetVersion: v2.5.3
It doesn't reproduce on a fresh cluster starting in production 2.5.2. On the cluster in question, I resolved the issue by deleting the CR, and it was recreated with the correct observedVersion (2.5.3).
Could not reporoduce the issue: Tried to reproduce by scaling down virt-api. The error about ""unable to retrieve the complete list of server APIs" appears in SSP log but once the api is back up, the error stops.
Let's try automating the workaround on HCO side.
As per comment 12 and 13, Verifying this BZ
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Virtualization 2.5.6 Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2021:2045