Bug 2014509

Summary:	Provide direct upgrade path from 2.6.4 to the next 4.8.4 release
Product:	Container Native Virtualization (CNV)	Reporter:	Arvin Amirian <aamirian>
Component:	Installation	Assignee:	Simone Tiraboschi <stirabos>
Status:	CLOSED ERRATA	QA Contact:	Kedar Bidarkar <kbidarka>
Severity:	high	Docs Contact:
Priority:	high
Version:	2.6.4	CC:	cnv-qe-bugs, dbasunag, fdeutsch, ocohen, sgott, stirabos, ycui
Target Milestone:	---
Target Release:	4.8.4
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	hco-bundle-registry-container-v4.8.4-44	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-01-20 17:21:21 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2008511, 2037307
Bug Blocks:

Description Arvin Amirian 2021-10-15 12:48:53 UTC

Description of problem:

Morgan Stanley's OCP clusters are deployed in a disconnected fashion. Upgrading to the latest version of CNV is painful due to the number of hops required. This results in a ton of images that need to be scanned, approved as well as the complexity of automating the upgrade. To avoid multiple upgrade hops in a disconnected environment we want to request a direct upgrade path from 2.6.4 to the next 4.8.x release.

Comment 2 Simone Tiraboschi 2021-10-20 07:16:08 UTC

on v4.8.3 now we have:
skipRange: ">=2.6.7 <4.8.3"

Comment 3 Kedar Bidarkar 2021-10-20 14:35:31 UTC

[kbidarka@localhost manifests-iib-1634729884]$ oc get nodes
NAME                                STATUS   ROLES    AGE   VERSION
kbidfinal483-vp4kc-master-0         Ready    master   18h   v1.21.1+a620f50
kbidfinal483-vp4kc-master-1         Ready    master   19h   v1.21.1+a620f50
kbidfinal483-vp4kc-master-2         Ready    master   18h   v1.21.1+a620f50
kbidfinal483-vp4kc-worker-0-4d2jz   Ready    worker   18h   v1.21.1+a620f50
kbidfinal483-vp4kc-worker-0-pwx7d   Ready    worker   18h   v1.21.1+a620f50
kbidfinal483-vp4kc-worker-0-r54bv   Ready    worker   18h   v1.21.1+a620f50

[kbidarka@localhost ocp-cnv-scripts]$ oc get csv -n openshift-cnv 
NAME                                      DISPLAY                    VERSION   REPLACES                                  PHASE
kubevirt-hyperconverged-operator.v2.6.7   OpenShift Virtualization   2.6.7     kubevirt-hyperconverged-operator.v2.6.6   Succeeded

[kbidarka@localhost manifests-iib-1634729884]$ oc patch catsrc hco-catalogsource -n openshift-marketplace --type merge -p '{"spec":{"image":"registry-proxy.engineering.redhat.com/rh-osbs/iib:126249"}}'
catalogsource.operators.coreos.com/hco-catalogsource patched


[kbidarka@localhost manifests-iib-1634729884]$ oc get ip -A
NAMESPACE                 NAME            CSV                                         APPROVAL    APPROVED
openshift-cnv             install-dhfvg   kubevirt-hyperconverged-operator.v4.8.3     Manual      false
openshift-cnv             install-smd29   kubevirt-hyperconverged-operator.v2.6.7     Manual      true

[kbidarka@localhost ocp-cnv-scripts]$ oc patch installplan install-dhfvg --namespace=openshift-cnv --type=merge '--patch={"spec":{"approved":true}}'
installplan.operators.coreos.com/install-dhfvg patched

kbidarka@localhost ocp-cnv-scripts]$ oc get csv -n openshift-cnv 
NAME                                      DISPLAY                    VERSION   REPLACES                                  PHASE
kubevirt-hyperconverged-operator.v2.6.7   OpenShift Virtualization   2.6.7     kubevirt-hyperconverged-operator.v2.6.6   Replacing
kubevirt-hyperconverged-operator.v4.8.3   OpenShift Virtualization   4.8.3     kubevirt-hyperconverged-operator.v2.6.7   InstallReady

[kbidarka@localhost ocp-cnv-scripts]$ oc get csv -n openshift-cnv 
NAME                                      DISPLAY                    VERSION   REPLACES                                  PHASE
kubevirt-hyperconverged-operator.v2.6.7   OpenShift Virtualization   2.6.7     kubevirt-hyperconverged-operator.v2.6.6   Replacing
kubevirt-hyperconverged-operator.v4.8.3   OpenShift Virtualization   4.8.3     kubevirt-hyperconverged-operator.v2.6.7   Installing


[kbidarka@localhost ocp-cnv-scripts]$ oc get csv -n openshift-cnv 
NAME                                      DISPLAY                    VERSION   REPLACES                                  PHASE
kubevirt-hyperconverged-operator.v4.8.3   OpenShift Virtualization   4.8.3     kubevirt-hyperconverged-operator.v2.6.7   InstallReady


[kbidarka@localhost ocp-cnv-scripts]$ oc get csv -n openshift-cnv 
NAME                                      DISPLAY                    VERSION   REPLACES                                  PHASE
kubevirt-hyperconverged-operator.v4.8.3   OpenShift Virtualization   4.8.3     kubevirt-hyperconverged-operator.v2.6.7   Succeeded




Summary: 

1) We can now directly upgrade to CNV version 4.8.3 from CNV Version 2.6.7
2) Virt-operator version after upgrade to 4.8.3: container-native-virtualization/virt-operator/images/v4.8.3-6
3) Virt-Launcher version after upgrade to 4.8.3: container-native-virtualization/virt-launcher/images/v4.8.3-6

Comment 5 Simone Tiraboschi 2021-10-25 15:54:18 UTC

We will enable a direct upgrade path: 2.6.4 -> 4.8.3

Comment 6 Simone Tiraboschi 2021-10-25 15:54:41 UTC

We will enable a direct upgrade path: 2.6.4 -> 4.8.4, sorry

Comment 8 Simone Tiraboschi 2022-01-04 10:36:47 UTC

@aamirian FYI:

4.8.4 bundle now correctly set skips: 2.6.4 to enable a direct 2.6.4 -> 4.8.4
so if the customer, in a disconnected environment, mirror now new bits the OLM will correctly choose 4.8.4 instead of 2.6.5 (which replaces 2.6.4).

In the eventually that the customer already mirrored 2.6.5 in the past, the OLM can already has created the 2.6.5 install plan.
In that case the OLM is not going to automatically delete the 2.6.5 installplan to create the 4.8.4 one so in that case you will eventually need to manually delete it to let the OLM correctly create the 4.8.4 this time.

Comment 10 Oren Cohen 2022-01-07 11:47:39 UTC

https://bugzilla.redhat.com/show_bug.cgi?id=2037307 is ON_QA and the direct upgrade path from 2.6.4 to 4.8.4 has been re-enabled in CNV 4.8.4-44, which is now in staging.
Thus, moving this BZ to ON_QA as well.

Comment 11 Kedar Bidarkar 2022-01-07 17:20:02 UTC

Direct CNV upgrade from v2.6.4  to v4.8.4
--------------------------------------------
[kbidarka@localhost ocp-cnv-scripts]$ oc get ip -A
NAMESPACE                 NAME            CSV                                         APPROVAL    APPROVED
openshift-cnv             install-fw9n5   kubevirt-hyperconverged-operator.v2.6.4     Manual      true
openshift-cnv             install-v42hc   kubevirt-hyperconverged-operator.v4.8.4     Manual      true

Summary:
1) Was able to Directly Upgrade CNV from v2.6.4 to v4.8.4 successfully.
2) Had to manually delete v2.6.5 install-plan for v4.8.4 install-plan to appear. ( In sync with comment 8 above )

Also the following bug: https://bugzilla.redhat.com/show_bug.cgi?id=2037307 is verified.
VERIFIED with: virt-operator-container-v4.8.4-12 hco-bundle-registry-container-v4.8.4-44

Comment 21 errata-xmlrpc 2022-01-20 17:21:21 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Virtualization 4.8.4 Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:0213