Was originally titled: "Kernel and other packages older in 4.5.9 than 4.4.21" +++ This bug was initially created as a clone of Bug #1879260 +++ Root cause TBD, but the RHEL package set in 4.4.21 is newer than 4.5.9. https://releases-rhcos-art.cloud.privileged.psi.redhat.com/diff.html?arch=x86_64&first_release=44.82.202009091324-0&first_stream=releases%2Frhcos-4.4&second_release=45.82.202009040230-0&second_stream=releases%2Frhcos-4.5 This bug is mainly to provide reference for blocking the upgrade edge from 4.4.21 to 4.5.9 and will immediately go MODIFIED as this will be resolved in 4.5.10. --- Additional comment from errata-xmlrpc on 2020-09-15 19:45:06 UTC --- This bug has been added to advisory RHBA-2020:3719 by Scott Dodson (sdodson) --- Additional comment from errata-xmlrpc on 2020-09-15 19:45:19 UTC --- Bug report changed to ON_QA status by Errata System. A QE request has been submitted for advisory RHBA-2020:3719-02 https://errata.devel.redhat.com/advisory/59084
The root cause appears to be the timing of when the z-stream release payloads were created. Release 4.5.9 was created from registry.svc.ci.openshift.org/ocp/release:4.5.0-0.nightly-2020-09-07-062006 Release 4.4.21 was created from registry.svc.ci.openshift.org/ocp/release:4.4.0-0.nightly-2020-09-09-153044 RHEL 8.2 batch 3 content was released on September 8. Since all supported/released RHCOS streams are built from the same RHEL repo, when the repo content is detected as changed, all the RHCOS versions should be built around the same time. In this case, the 4.5.9 release (created Sep 7) used RHCOS 45.82.202009040230-0 (built on Sep 4). When the 4.4.21 release was created (Sep 9), the RHCOS used was 44.82.202009091324-0 (built on Sep 9). What happened in between the creation of the two release payloads was the release of the RHEL 8.2 batch content on Sep 8. This means that all the RHCOS versions were updated on that day; see below for the 4.3, 4.4, and 4.5 builds that correspond to this content change: https://releases-rhcos-art.cloud.privileged.psi.redhat.com/?stream=releases/rhcos-4.3&release=43.82.202009080953.0#43.82.202009080953.0 https://releases-rhcos-art.cloud.privileged.psi.redhat.com/?stream=releases/rhcos-4.4&release=44.82.202009080930-0#44.82.202009080930-0 https://releases-rhcos-art.cloud.privileged.psi.redhat.com/?stream=releases/rhcos-4.5&release=45.82.202009081029-0#45.82.202009081029-0 What did *not* happen was an updated release for 4.5.z, so we end up with the older version (4.4.21) with newer RHEL content. Some suggestions that were made for addressing this: - pause the RHCOS build pipelines when ART does their first z-stream release of the week until the final z-stream is created - pause the machine-os-content promote job when ART does their first z-stream release of the week until the final z-stream is created (I tend to favor the latter, as the RHCOS build pipeline could still do basic sanity testing via qemu and wouldn't be blind to changes for a few days)
Higher priority work has prevented action on this; setting UpcomingSprint
Higher priority work has prevented this issue from being solved; adding the UpcomingSprint keyword