Description of problem: As part of migrating k8s-nmstate from CNV to be a standalone OCP operator (https://issues.redhat.com/browse/CNV-7568), upgrading CNV 4.10.1 to CNV 4.11 should be blocked if k8s-nmstate is still installed as part of CNV (i.e. its components, e.g. nmstate-handler DaemonSet, are installed in openshift-cnv namespace). Version-Release number of selected component (if applicable): 4.10.1 How reproducible: Tried only once until now Steps to Reproduce: 1. In a cluster with OCP 4.11 (after upgrading from 4.10) and CNV 4.10.1 - try upgrading CNV to 4.11 (instructions in the following comment, which will be private). Actual results: Upgrade succeeds, and CNV 4.11 is installed. Expected results: Upgrade should be blocked until CNV knmstate is removed, and standalone knmstate operator (KNO) is installed. Additional info:
On a 4.10.2 cluster post upgrade of ocp, I see this: ==================================================== [cnv-qe-jenkins@c01-dbn-4012-zxn69-executor ~]$ kubectl get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq ".status.versions" [ { "name": "operator", "version": "4.10.2" } ] [cnv-qe-jenkins@c01-dbn-4012-zxn69-executor ~]$ kubectl get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq ".status.conditions" [ { "lastTransitionTime": "2022-05-31T21:29:30Z", "message": "Reconcile completed successfully", "observedGeneration": 2, "reason": "ReconcileCompleted", "status": "True", "type": "ReconcileComplete" }, { "lastTransitionTime": "2022-06-03T10:27:01Z", "message": "Reconcile completed successfully", "observedGeneration": 2, "reason": "ReconcileCompleted", "status": "True", "type": "Available" }, { "lastTransitionTime": "2022-06-03T10:27:01Z", "message": "Reconcile completed successfully", "observedGeneration": 2, "reason": "ReconcileCompleted", "status": "False", "type": "Progressing" }, { "lastTransitionTime": "2022-06-03T10:27:01Z", "message": "Reconcile completed successfully", "observedGeneration": 2, "reason": "ReconcileCompleted", "status": "False", "type": "Degraded" }, { "lastTransitionTime": "2022-06-02T22:00:02Z", "message": "NetworkAddonsConfig is not upgradeable: NMState deployment is not supported by CNAO anymore, please install Kubernetes NMState Operator", "observedGeneration": 2, "reason": "NetworkAddonsConfigNotUpgradeable", "status": "False", "type": "Upgradeable" } ] [cnv-qe-jenkins@c01-dbn-4012-zxn69-executor ~]$ ====================== installed kubernetes nmstate: ====================== [cnv-qe-jenkins@c01-dbn-4012-zxn69-executor ~]$ kubectl get all -n openshift-nmstate I0603 11:46:35.420733 96309 request.go:665] Waited for 1.143379287s due to client-side throttling, not priority and fairness, request: GET:https://api.c01-dbn-4012.cnv-qe.rhcloud.com:6443/apis/imageregistry.operator.openshift.io/v1?timeout=32s NAME READY STATUS RESTARTS AGE pod/nmstate-cert-manager-5949c9b46d-t9tfr 1/1 Running 0 89s pod/nmstate-handler-4pxtr 1/1 Running 0 88s pod/nmstate-handler-hndx7 1/1 Running 0 88s pod/nmstate-handler-k2ttg 1/1 Running 0 88s pod/nmstate-handler-k485z 1/1 Running 0 89s pod/nmstate-handler-m8dj5 1/1 Running 0 88s pod/nmstate-handler-swkf8 1/1 Running 0 88s pod/nmstate-operator-7646b78f5f-fgl5t 1/1 Running 0 2m5s pod/nmstate-webhook-f965464-89frq 1/1 Running 0 89s pod/nmstate-webhook-f965464-znkmz 1/1 Running 0 89s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/nmstate-webhook ClusterIP 172.30.86.205 <none> 443/TCP 89s NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/nmstate-handler 6 6 6 6 6 beta.kubernetes.io/arch=amd64,kubernetes.io/os=linux 89s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/nmstate-cert-manager 1/1 1 1 89s deployment.apps/nmstate-operator 1/1 1 1 2m5s deployment.apps/nmstate-webhook 2/2 2 2 89s NAME DESIRED CURRENT READY AGE replicaset.apps/nmstate-cert-manager-5949c9b46d 1 1 1 89s replicaset.apps/nmstate-operator-7646b78f5f 1 1 1 2m5s replicaset.apps/nmstate-webhook-f965464 2 2 2 89s [cnv-qe-jenkins@c01-dbn-4012-zxn69-executor ~]$ ============================== Operator condition changed back to upgradable: ============================== status: conditions: - lastTransitionTime: "2022-06-03T11:45:07Z" message: "" observedGeneration: 7 reason: Upgradeable status: "True" type: Upgradeable ============================== HCO condition stabilized: ============================== [cnv-qe-jenkins@c01-dbn-4012-zxn69-executor ~]$ kubectl get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq ".status.conditions" [ { "lastTransitionTime": "2022-05-31T21:29:30Z", "message": "Reconcile completed successfully", "observedGeneration": 2, "reason": "ReconcileCompleted", "status": "True", "type": "ReconcileComplete" }, { "lastTransitionTime": "2022-06-03T11:45:07Z", "message": "Reconcile completed successfully", "observedGeneration": 2, "reason": "ReconcileCompleted", "status": "True", "type": "Available" }, { "lastTransitionTime": "2022-06-03T10:27:01Z", "message": "Reconcile completed successfully", "observedGeneration": 2, "reason": "ReconcileCompleted", "status": "False", "type": "Progressing" }, { "lastTransitionTime": "2022-06-03T11:45:07Z", "message": "Reconcile completed successfully", "observedGeneration": 2, "reason": "ReconcileCompleted", "status": "False", "type": "Degraded" }, { "lastTransitionTime": "2022-06-03T11:45:07Z", "message": "Reconcile completed successfully", "observedGeneration": 2, "reason": "ReconcileCompleted", "status": "True", "type": "Upgradeable" } ] After this cnv upgrade to 4.11 completed successfully: [cnv-qe-jenkins@c01-dbn-4012-zxn69-executor ~]$ kubectl get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq ".status.versions" [ { "name": "operator", "version": "4.11.0" } ] [cnv-qe-jenkins@c01-dbn-4012-zxn69-executor ~]$
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Virtualization 4.10.2 Images security and bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5026