Description of problem: Since we have bug in migration https://bugzilla.redhat.com/show_bug.cgi?id=2013494 We can't upgrade old version: 2.5.8, 2.6.4,2.6.5,2.6.6 To CNV 2.6.7 We will need to skip this version and direct to 2.6.8 Note that we will need to also make sure that we will upgrade from 2.6.8 to 4.8.3 not to hit the migration bug https://bugzilla.redhat.com/show_bug.cgi?id=2013494. Version-Release number of selected component (if applicable): 2.6.7 How reproducible: 100% Additional info: Verification flow of the migration bug Start with VM in 2.5.8 (CNV 2.5.8, OCP 4.6) Do OCP upgrade to 4.7 and start CNV upgrade with the following scenarios: Scenario 1: Upgrade CNV From: 2.5.8 To: 2.6.4 Migration VM in CNV 2.6.4 (Virt-Launcher version - 2.6.4/2.6.3-2): migration succeed Continue the upgrade to CNV 2.6.8(Virt-Launcher version 2.6.4/2.6.3-2 to 2.6.8-5) - PASSED Scenario 2: Upgrade CNV From: 2.5.8 To: 2.6.5 Migration VM in CNV 2.6.5(Virt-Launcher version - 2.6.5-2): migration succeed Continue the upgrade to CNV 2.6.8(Virt-Launcher version 2.6.5-2 to 2.6.8-5) - PASSED Scenario 3: Upgrade CNV From: 2.5.8 To: 2.6.6 Migration VM in CNV 2.6.6(Virt-Launcher version - 2.6.6-7): migration succeed Continue the upgrade to CNV 2.6.8(Virt-Launcher version 2.6.6-7 to 2.6.8-5) - PASSED Scenario 4: Upgrade CNV From: 2.5.8 To: 2.6.7 Migration VM in CNV 2.6.7() migration Failed - hit the bug: (https://bugzilla.redhat.com/show_bug.cgi?id=2013494) [CNV-2.6.8] VMI is in LiveMigrate loop when Upgrading Cluster from 2.6.7/4.7.32 to OCP 4.8.13 Since this bug: We stopped and start the VM and continue the upgrade to CNV 2.6.8 - PASSED Scenario 5: Upgrade CNV From: 2.5.8 To: 2.6.8 Continue the upgrade to CNV 2.6.8 - PASSED Migration VM in CNV 2.6.8 migration succeed
Scenario 4: Upgrade CNV From: 2.5.8 To: 2.6.7 --- Migrate VM in CNV 2.6.7 LiveMigration - FAILED Virt-Launcher version 2.5.8 to 2.6.7 Source Virt-Launcher Pod 2.5.8 continues to be in Running state. Target Virt-launcher Pod 2.6.7 enters Completed or Error state VMIM Object shows Status: FAILED We hit the bug: (https://bugzilla.redhat.com/show_bug.cgi?id=2013494) Continued upgrade to CNV 2.6.8 Upgraded the same VMI/Virt-Launcher Pod, version 2.5.8 to 2.6.8-5 - PASSED ---
I just set: olm.skipRange: '>=2.5.8 <2.6.0 || >=2.6.6 <2.6.8' it will be available in the next build of the bundle. QE should explicitly test: - 2.5.8 => 2.6.8 - 2.6.6 => 2.6.8 - 2.6.7 => 2.6.8 @ipinto please ack
(In reply to Simone Tiraboschi from comment #2) > I just set: > olm.skipRange: '>=2.5.8 <2.6.0 || >=2.6.6 <2.6.8' > > it will be available in the next build of the bundle. > > QE should explicitly test: > - 2.5.8 => 2.6.8 > - 2.6.6 => 2.6.8 > - 2.6.7 => 2.6.8 > > @ipinto please ack Debarati: Please verify the skip range. Simone: WE should also have 2.6.4 => 2.6.8 No?
(In reply to Israel Pinto from comment #3) > Simone: > WE should also have 2.6.4 => 2.6.8 No? No
@ipinto Validated olm.skipRange is set right against 2.6.3-23 =========================== [cnv-qe-jenkins@c01-besso-26-pqdzz-executor ~]$ kubectl get csv -n openshift-cnv kubevirt-hyperconverged-operator.v2.6.8 -o yaml apiVersion: operators.coreos.com/v1alpha1 kind: ClusterServiceVersion metadata: annotations: alm-examples: '[{"apiVersion":"hco.kubevirt.io/v1beta1","kind":"HyperConverged","metadata":{"annotations":{"deployOVS":"false"},"name":"kubevirt-hyperconverged","namespace":"openshift-cnv"},"spec":{"BareMetalPlatform":false}},{"apiVersion":"networkaddonsoperator.network.kubevirt.io/v1","kind":"NetworkAddonsConfig","metadata":{"name":"cluster"},"spec":{"imagePullPolicy":"IfNotPresent","kubeMacPool":{"rangeEnd":"FD:FF:FF:FF:FF:FF","rangeStart":"02:00:00:00:00:00"},"linuxBridge":{},"macvtap":{},"multus":{},"nmstate":{},"ovs":{}}},{"apiVersion":"kubevirt.io/v1alpha3","kind":"KubeVirt","metadata":{"name":"kubevirt","namespace":"kubevirt"},"spec":{"imagePullPolicy":"Always"}},{"apiVersion":"ssp.kubevirt.io/v1beta1","kind":"SSP","metadata":{"name":"ssp-sample","namespace":"kubevirt"},"spec":{"commonTemplates":{"namespace":"kubevirt"},"templateValidator":{"replicas":2}}},{"apiVersion":"cdi.kubevirt.io/v1beta1","kind":"CDI","metadata":{"name":"cdi","namespace":"cdi"},"spec":{"imagePullPolicy":"IfNotPresent"}},{"apiVersion":"nodemaintenance.kubevirt.io/v1beta1","kind":"NodeMaintenance","metadata":{"name":"nodemaintenance-example"},"spec":{"nodeName":"node02","reason":"Test node maintenance"}},{"apiVersion":"hostpathprovisioner.kubevirt.io/v1beta1","kind":"HostPathProvisioner","metadata":{"name":"hostpath-provisioner"},"spec":{"imagePullPolicy":"IfNotPresent","pathConfig":{"path":"/var/hpvolumes","useNamingPrefix":false}}},{"apiVersion":"v2v.kubevirt.io/v1beta1","kind":"VMImportConfig","metadata":{"name":"vm-import-operator-config"},"spec":{"imagePullPolicy":"IfNotPresent"}}]' capabilities: Full Lifecycle categories: OpenShift Optional certified: "true" containerImage: registry.redhat.io/container-native-virtualization/hyperconverged-cluster-operator@sha256:21a379643b694951d1c703c76c4e4fb28f40b79900d1b2956633e8ef2818c3fe createdAt: "2021-11-04 10:17:44" description: Creates and maintains an OpenShift Virtualization Deployment olm.operatorGroup: kubevirt-hyperconverged-group olm.operatorNamespace: openshift-cnv olm.properties: '[{"type": "olm.maxOpenShiftVersion", "value": "4.8"}]' olm.skipRange: '>=2.5.8 <2.6.0 || >=2.6.6 <2.6.8' olm.targetNamespaces: openshift-cnv =========================== [cnv-qe-jenkins@c01-besso-26-pqdzz-executor ~]$ kubectl get pods -n openshift-cnv hco-operator-855f8b96b6-wrn4n -o yaml | grep olm.skipRange olm.skipRange: '>=2.5.8 <2.6.0 || >=2.6.6 <2.6.8' [cnv-qe-jenkins@c01-besso-26-pqdzz-executor ~]$ kubectl get pods -n openshift-cnv hco-webhook-78457c84fc-x2d9t -o yaml | grep olm.skipRange olm.skipRange: '>=2.5.8 <2.6.0 || >=2.6.6 <2.6.8' [cnv-qe-jenkins@c01-besso-26-pqdzz-executor ~]$ =========================== Not marking this as verified as following upgrade is yet to complete: =========================== 3) 2.5.8 to 2.6.8
On an upgraded cluster: ====================== [cnv-qe-jenkins@c01-258upgr-225h6-executor ~]$ kubectl get csv -A I1115 15:43:04.369241 81927 request.go:665] Waited for 1.157086389s due to client-side throttling, not priority and fairness, request: GET:https://api.c01-258upgr.cnv-qe.rhcloud.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s NAMESPACE NAME DISPLAY VERSION REPLACES PHASE openshift-cnv kubevirt-hyperconverged-operator.v2.6.8 OpenShift Virtualization 2.6.8 kubevirt-hyperconverged-operator.v2.5.8 Succeeded openshift-local-storage local-storage-operator.4.7.0-202110121415 Local Storage 4.7.0-202110121415 Succeeded openshift-operator-lifecycle-manager packageserver Package Server 0.17.0 Succeeded openshift-storage ocs-operator.v4.7.5 OpenShift Container Storage 4.7.5 ocs-operator.v4.7.4 Succeeded [cnv-qe-jenkins@c01-258upgr-225h6-executor ~]$ ====================== [cnv-qe-jenkins@c01-258upgr-225h6-executor ~]$ kubectl get csv -n openshift-cnv kubevirt-hyperconverged-operator.v2.6.8 -o yaml apiVersion: operators.coreos.com/v1alpha1 kind: ClusterServiceVersion metadata: annotations: alm-examples: '[{"apiVersion":"hco.kubevirt.io/v1beta1","kind":"HyperConverged","metadata":{"annotations":{"deployOVS":"false"},"name":"kubevirt-hyperconverged","namespace":"openshift-cnv"},"spec":{"BareMetalPlatform":false}},{"apiVersion":"networkaddonsoperator.network.kubevirt.io/v1","kind":"NetworkAddonsConfig","metadata":{"name":"cluster"},"spec":{"imagePullPolicy":"IfNotPresent","kubeMacPool":{"rangeEnd":"FD:FF:FF:FF:FF:FF","rangeStart":"02:00:00:00:00:00"},"linuxBridge":{},"macvtap":{},"multus":{},"nmstate":{},"ovs":{}}},{"apiVersion":"kubevirt.io/v1alpha3","kind":"KubeVirt","metadata":{"name":"kubevirt","namespace":"kubevirt"},"spec":{"imagePullPolicy":"Always"}},{"apiVersion":"ssp.kubevirt.io/v1beta1","kind":"SSP","metadata":{"name":"ssp-sample","namespace":"kubevirt"},"spec":{"commonTemplates":{"namespace":"kubevirt"},"templateValidator":{"replicas":2}}},{"apiVersion":"cdi.kubevirt.io/v1beta1","kind":"CDI","metadata":{"name":"cdi","namespace":"cdi"},"spec":{"imagePullPolicy":"IfNotPresent"}},{"apiVersion":"nodemaintenance.kubevirt.io/v1beta1","kind":"NodeMaintenance","metadata":{"name":"nodemaintenance-example"},"spec":{"nodeName":"node02","reason":"Test node maintenance"}},{"apiVersion":"hostpathprovisioner.kubevirt.io/v1beta1","kind":"HostPathProvisioner","metadata":{"name":"hostpath-provisioner"},"spec":{"imagePullPolicy":"IfNotPresent","pathConfig":{"path":"/var/hpvolumes","useNamingPrefix":false}}},{"apiVersion":"v2v.kubevirt.io/v1beta1","kind":"VMImportConfig","metadata":{"name":"vm-import-operator-config"},"spec":{"imagePullPolicy":"IfNotPresent"}}]' capabilities: Full Lifecycle categories: OpenShift Optional certified: "true" containerImage: registry.redhat.io/container-native-virtualization/hyperconverged-cluster-operator@sha256:21a379643b694951d1c703c76c4e4fb28f40b79900d1b2956633e8ef2818c3fe createdAt: "2021-11-04 10:17:44" description: Creates and maintains an OpenShift Virtualization Deployment olm.operatorGroup: kubevirt-hyperconverged-group olm.operatorNamespace: openshift-cnv olm.properties: '[{"type": "olm.maxOpenShiftVersion", "value": "4.8"}]' olm.skipRange: '>=2.5.8 <2.6.0 || >=2.6.6 <2.6.8' olm.targetNamespaces: openshift-cnv ==================================== [cnv-qe-jenkins@c01-258upgr-225h6-executor ~]$ kubectl get pods -n openshift-cnv hco-operator-f6498bb64-vzd57 -o yaml | grep olm.skipRange olm.skipRange: '>=2.5.8 <2.6.0 || >=2.6.6 <2.6.8' [cnv-qe-jenkins@c01-258upgr-225h6-executor ~]$ kubectl get pods -n openshift-cnv hco-webhook-5c86fff6fc-zgs4g -o yaml | grep olm.skipRange olm.skipRange: '>=2.5.8 <2.6.0 || >=2.6.6 <2.6.8' [cnv-qe-jenkins@c01-258upgr-225h6-executor ~]$
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days