Hide Forgot
+++ This bug was initially created as a clone of Bug #1999556 +++ We're seeing instances of the following error still: the "master" pool should be updated before the CVO reports available at the new version occurred It looks like maybe a regression of https://bugzilla.redhat.com/show_bug.cgi?id=1970150? See: https://search.ci.openshift.org/?search=pool+should+be+updated+before+the+CVO+reports+available+at+the+new+version&maxAge=168h&context=1&type=bug%2Bjunit&name=4.9&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job --- Additional comment from Kirsten Garrison on 2021-09-01 23:45:17 UTC --- I'll take a look at this since I worked on the previous fix and want to figure out what's happening here. --- Additional comment from Kirsten Garrison on 2021-09-02 02:11:54 UTC --- timeline from https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.9-e2e-aws-upgrade/1432377737405796352 17:42:20 default machineconfigoperator machine-config OperatorVersionChanged clusteroperator/machine-config-operator started a version change from [{operator 4.9.0-0.nightly-2021-08-30-070917}] to [{operator 4.9.0-0.nightly-2021-08-30-161832}] - lastTransitionTime: "2021-08-30T17:42:49Z" message: sync completed towards (2) generation using controller version v4.9.0-202108281318.p0.git.00f349e.assembly.stream-dirty status: "True" type: TemplateControllerCompleted Same osimageurl, same controller version: machineconfiguration.openshift.io/generated-by-version: v4.9.0-202108281318.p0.git.00f349e.assembly.stream-dirty 17:42:51 default machineconfigoperator machine-config OperatorVersionChanged clusteroperator/machine-config-operator version changed from [{operator 4.9.0-0.nightly-2021-08-30-070917}] to [{operator 4.9.0-0.nightly-2021-08-30-161832}] 17:42:53 default machineconfigcontroller-rendercontroller master RenderedConfigGenerated rendered-master-5d8b2493d5d9fd8e6a6762f985bf5828 successfully generated This is only happening intermittently probably some sort of timing esp for the metal platform, looks like we'll have to harden this up more. Likely verify checks against release version. TBD. --- Additional comment from Scott Dodson on 2021-11-18 15:44:27 UTC --- This issue comes up as a late emergency debugging situation anytime we ship a z-stream which hasn't updated the MCO and osImageURL (suspected). It happens in frequently enough that everyone forgets about this bug and sinks substantial amount of effort into figuring out what's wrong at the 11th hour before we ship a release. As such I'm going to mark this as blocker+ so that we can avoid that firedrill in the future. Once fixed we should backport this to all currently supported releases unless there's a technical reason not to.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.8.34 bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:0795