+++ This bug was initially created as a clone of Bug #2097555 +++ Description of problem: The cluster-ingress-operator is causing 4.9 -> 4.10 CI upgrade tests to fail: { fail [github.com/onsi/ginkgo.0-origin.0+incompatible/internal/leafnodes/runner.go:113]: Jun 15 18:22:34.283: Some cluster operators are not ready: ingress (Upgradeable=False IngressControllersNotUpgradeable: Some ingresscontrollers are not upgradeable: ingresscontroller "default" is not upgradeable: OperandsNotUpgradeable: One or more managed resources are not upgradeable: load balancer service has been modified; changes must be reverted before upgrading: )} OpenShift release version: 4.9 Cluster Platform: GCP, OVN How reproducible: Unclear, but most recent nightly GCP OVN upgrade tests are failing due to this bug, even with three retries per nightly run. Steps to Reproduce (in detail): 1. Run OCP 4.9 to 4.10 nightly upgrade test Actual results: OCP cluster upgrade fails Expected results: OCP cluster upgrade fails Impact of the problem: This is currently blocking releases in the 4.10.z stream. Additional info: Examples of failing runs: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.10-upgrade-from-stable-4.9-e2e-gcp-ovn-upgrade/1537128423896387584 https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.10-upgrade-from-stable-4.9-e2e-gcp-ovn-upgrade/1536843529286848512
I was able to reproduce the failure using Cluster Bot and the command "test upgrade 4.9 4.10 azure,ovn". Here is the job: <https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-origin-installer-launch-azure/1537476242792517632 Here is the failure: Jun 16 17:31:04.782: FAIL: Some cluster operators are not ready: ingress (Upgradeable=False IngressControllersNotUpgradeable: Some ingresscontrollers are not upgradeable: ingresscontroller "default" is not upgradeable: OperandsNotUpgradeable: One or more managed resources are not upgradeable: load balancer service has been modified; changes must be reverted before upgrading: ) I was able to verify the fix using Cluster Bot and the command "test upgrade 4.9,openshift/cluster-ingress-operator#784 4.10,openshift/cluster-ingress-operator#784 azure,ovn". Here is the job: <https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-origin-installer-launch-azure/1537490526649978880>. The "IngressControllersNotUpgradeable" failure does not appear.
*** Bug 2099663 has been marked as a duplicate of this bug. ***
Verified upgrading 4.9.0-0.nightly-2022-06-24-070308 to 4.10.0-0.nightly-2022-07-06-083652 twice on GCP. Both upgrades were successful. @hongli Not sure if you want to do any other testing around this.
and verified that ingress Upgradeable=True $ oc get co/ingress NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE ingress 4.10.0-0.nightly-2022-07-06-083652 True False False 3h59m $ oc get co/ingress -oyaml <---snip---> - lastTransitionTime: "2022-07-07T03:50:50Z" reason: IngressControllersUpgradeable status: "True" type: Upgradeable
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.10.23 bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:5568