Bug 2110783

Summary: 4.8 to 4.9 upgrade path blocked due to bug in NMState pods
Product: Container Native Virtualization (CNV) Reporter: Arvin Amirian <aamirian>
Component: InstallationAssignee: Simone Tiraboschi <stirabos>
Status: CLOSED ERRATA QA Contact: Satyajit Bulage <sbulage>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 4.9.5CC: cnv-qe-bugs, kmajcher, kmcdonald, phoracek, stirabos
Target Milestone: ---   
Target Release: 4.9.6   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: hco-bundle-registry-container-v4.9.6-21 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2111075 (view as bug list) Environment:
Last Closed: 2022-09-22 08:17:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2110905    
Bug Blocks: 2111075    

Description Arvin Amirian 2022-07-25 21:15:01 UTC
Description of problem:
As described in the following bugzilla, NMState pods not compatible with Calico CNI provider preventing deploy of Openshift Virtualization. 
https://bugzilla.redhat.com/show_bug.cgi?id=2078247

This was fixed in 4.10 but the fix needs to be backported to 4.9 to allow upgrades from 4.8 to 4.10

Version-Release number of selected component (if applicable):
4.9.5

How reproducible:
Every time

Steps to Reproduce:
1. Deploy or upgrade OpenShift Virtualization 4.9 to any cluster with Calico CNI

Actual results:
Deploy and upgrade fail due to nmstate pod

Expected results:
Deploy and Upgrades are successfull

Additional info:

Previously upgrades paths were created for client to upgrade from 4.8.4 -> 4.9.5 -> 4.10.3


Once this fix is done the same upgrade path should be open for the new releases. 4.8.4 -> 4.9.6 -> 4.10.4

Comment 2 Satyajit Bulage 2022-08-10 14:44:03 UTC
Direct path exists between 4.8.4 to 4.9.6.

Upgrade Result:
---------------
[cnv-qe-jenkins@c01-satya-48u-hd5tg-executor cnv-iib-manifests]$ oc get csv
NAME                                      DISPLAY                    VERSION   REPLACES                                  PHASE
kubevirt-hyperconverged-operator.v4.9.6   OpenShift Virtualization   4.9.6     kubevirt-hyperconverged-operator.v4.8.4   Succeeded


[cnv-qe-jenkins@c01-satya-48u-hd5tg-executor cnv-iib-manifests]$ oc get pod
NAME                                                   READY   STATUS    RESTARTS   AGE
bridge-marker-42886                                    1/1     Running   0          65m
bridge-marker-58gs8                                    1/1     Running   0          65m
bridge-marker-6jb6k                                    1/1     Running   0          65m
bridge-marker-6nwhj                                    1/1     Running   0          65m
bridge-marker-6q74z                                    1/1     Running   0          65m
bridge-marker-bkx22                                    1/1     Running   0          65m
cdi-apiserver-7b99756878-s6m47                         1/1     Running   0          8m22s
cdi-deployment-55c469499f-qxpd4                        1/1     Running   0          8m21s
cdi-operator-66bff88458-rck5v                          1/1     Running   0          8m21s
cdi-uploadproxy-b8576f468-p7p2b                        1/1     Running   0          8m20s
cluster-network-addons-operator-788f6fbbd4-nt7t8       1/1     Running   0          8m22s
hco-operator-78bcfc9f4b-hmshj                          1/1     Running   0          8m22s
hco-webhook-b7459554c-zqdgk                            1/1     Running   0          8m26s
hostpath-provisioner-8rx2x                             1/1     Running   0          67m
hostpath-provisioner-gsf7m                             1/1     Running   0          67m
hostpath-provisioner-k7wnt                             1/1     Running   0          67m
hostpath-provisioner-operator-66bff6c84d-zq265         1/1     Running   0          8m23s
hyperconverged-cluster-cli-download-67dcf98d5b-7cwh4   1/1     Running   0          8m19s
kube-cni-linux-bridge-plugin-5pk2w                     1/1     Running   0          62m
kube-cni-linux-bridge-plugin-9qf6w                     1/1     Running   0          63m
kube-cni-linux-bridge-plugin-c2hww                     1/1     Running   0          62m
kube-cni-linux-bridge-plugin-lt8x9                     1/1     Running   0          64m
kube-cni-linux-bridge-plugin-rf5ml                     1/1     Running   0          64m
kube-cni-linux-bridge-plugin-vnlkj                     1/1     Running   0          65m
kubemacpool-cert-manager-87556b5d6-85gsv               1/1     Running   0          8m23s
kubemacpool-mac-controller-manager-5559d66bb5-jrzds    1/1     Running   0          5m49s
nmstate-cert-manager-6f5bf6785c-m2lsd                  1/1     Running   0          5m49s
nmstate-handler-6s2cq                                  1/1     Running   0          65m
nmstate-handler-dwmhg                                  1/1     Running   0          65m
nmstate-handler-k5k6k                                  1/1     Running   0          65m
nmstate-handler-lt7gc                                  1/1     Running   0          65m
nmstate-handler-q5rjs                                  1/1     Running   0          65m
nmstate-handler-wthf5                                  1/1     Running   0          65m
nmstate-webhook-54f5d567f7-sd6w2                       1/1     Running   0          8m41s
nmstate-webhook-54f5d567f7-xtgx6                       1/1     Running   0          5m48s
node-maintenance-operator-86586596c-62bwb              1/1     Running   0          5m49s
ssp-operator-5f9f5ffdd5-m9l46                          1/1     Running   0          8m20s
virt-api-96ff87955-lj8vg                               1/1     Running   0          8m19s
virt-api-96ff87955-wsn7l                               1/1     Running   0          6m32s
virt-controller-587b897fd5-kkcfz                       1/1     Running   0          6m31s
virt-controller-587b897fd5-xf94w                       1/1     Running   0          8m19s
virt-handler-4cqhq                                     1/1     Running   0          64m
virt-handler-6npvx                                     1/1     Running   0          65m
virt-handler-rn6hd                                     1/1     Running   0          65m
virt-operator-68ff44b7d4-c4679                         1/1     Running   0          8m19s
virt-operator-68ff44b7d4-clsd8                         1/1     Running   0          6m33s
virt-template-validator-77cf4c9796-2fcbt               1/1     Running   0          6m33s
virt-template-validator-77cf4c9796-kgt6c               1/1     Running   0          8m19s
vm-import-controller-5cdc759f74-rfmvx                  1/1     Running   0          6m33s
vm-import-operator-69549f7995-blbsv                    1/1     Running   0          6m32s

Comment 8 errata-xmlrpc 2022-09-22 08:17:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Virtualization 4.9.6 Images security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:6681