Hide Forgot
When there is no new MCO commit but there is an osimageurl change the master pool is still upgrading when the MCO reports level to the CVO. This is a copy of Bug #1955929, which seems to address the issue, however there are some failing runs related to https://bugzilla.redhat.com/show_bug.cgi?id=1968754 so using this BZ to carry the fix which drastically reduced failures and keeping the other BZ open to audit after the new metal-ipi bug is fixed. This bug was initially created as a copy of Bug #1955929 May 1 01:39:28.369: INFO: cluster upgrade is Progressing: Working towards 4.8.0-0.nightly-2021-05-01-000412: 652 of 675 done (96% complete) May 1 01:39:38.369: INFO: Completed upgrade to registry.build01.ci.openshift.org/ci-op-ns22yv9h/release@sha256:1aeba3cfeb93d5912390fbffafaa3d024ae8db26489b01b2fa034d421f69b5db May 1 01:39:38.460: INFO: Waiting on pools to be upgraded May 1 01:39:38.632: INFO: Pool master is still reporting (Updated: false, Updating: true, Degraded: false) May 1 01:39:38.632: INFO: Invariant violation detected: the "master" pool should be updated before the CVO reports available at the new version https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.8-e2e-aws-upgrade/1388283995501891584 Urgent because itβs happened in 38% of the last 16 upgrade jobs in nightly https://search.ci.openshift.org/?search=Pool+master+is+still+reporting&maxAge=48h&context=1&type=build-log&name=upgrade&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job
Created attachment 1789861 [details] upgrade progression 1
Created attachment 1789863 [details] upgrade progression 2
Created attachment 1789864 [details] upgrade progression 3
Created attachment 1789865 [details] upgrade progression 4
Created attachment 1789866 [details] upgrade progression 5
Verified on registry.ci.openshift.org/ocp/release:4.8.0-0.nightly-2021-06-10-014052. Upgraded to registry.ci.openshift.org/ocp/release:4.8.0-0.nightly-2021-06-10-045932 which has no new MCO commit and a new osImageURL. Watched `oc get co/machine-config` `oc get clusterversion` `oc get mcp`. Verified the `co/machine-config` did not transition to the new version until the master pool completed updating. See attachments.
*** Bug 1955929 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2438