Bug 2059762

Summary: 4.9.23-s390x-machine-os-content manifest invalid when mirroring content for disconnected install
Product: OpenShift Container Platform Reporter: Vikas Laad <vlaad>
Component: RHCOSAssignee: Colin Walters <walters>
Status: CLOSED ERRATA QA Contact: Michael Nguyen <mnguyen>
Severity: urgent Docs Contact:
Priority: high    
Version: 4.9CC: aos-bugs, chanphil, christian.lapolt, danili, dornelas, fleber, hhei, Holger.Wolf, jligon, krmoser, lmeyer, madeel, miabbott, mrussell, nstielau, scuppett, vlaad, walters, wking, wsun
Target Milestone: ---   
Target Release: 4.9.z   
Hardware: s390x   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 2059761
: 2059763 (view as bug list) Environment:
Last Closed: 2022-03-16 11:39:24 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2059761    
Bug Blocks: 2059763    

Comment 1 Micah Abbott 2022-03-04 17:31:28 UTC
The fix for `coreos-assembler:rhcos-4.9` landed in https://github.com/coreos/coreos-assembler/pull/2728

We are awaiting new 4.9 builds across all arches to be pushed to Quay in the v2s2 format

Comment 2 Colin Walters 2022-03-09 13:45:58 UTC
To clarify, the steps are (multiplied for each architecture and branch):

- Verify that there is a coreos-assembler build for that release and architecture with the patch
- Force a build - see e.g. https://gitlab.cee.redhat.com/openshift-art/rhcos-upshift/-/merge_requests/355 for an example
  (To find the correct build to force, you can see it in the logs of the previous jenkins job)

Comment 3 Micah Abbott 2022-03-09 20:30:31 UTC
All the machine-os-content images for 4.9 and all arches are confirmed to be in v2s2 format

Comment 6 HuijingHei 2022-03-10 06:17:09 UTC
Change status to VERIFIED according to Comment 5

Comment 9 errata-xmlrpc 2022-03-16 11:39:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.9.24 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:0798

Comment 10 W. Trevor King 2022-03-24 18:32:05 UTC
Per comment 3, 4.9.24 only uses application/vnd.docker.distribution.manifest.v2+json, avoiding issues when mirror pipelines have stages that don't support OCI manifest.  Checking 4.9.23, we had one OCI manifest (presumably for the machine-os-content image) on each of our four CPU architectures:

$ for ARCH in aarch64 ppc64le s390x x86_64; do echo "${ARCH}"; oc adm release info --pullspecs "quay.io/openshift-release-dev/ocp-release:4.9.23-${ARCH}" | sed -n 's|.*\(quay.io/\)|\1|p' | sort | uniq | 
while read IMAGE; do oc image info "${IMAGE}" | grep 'Media Type'; done | sort | uniq -c; done
aarch64
     11 Media Type:    application/vnd.docker.distribution.manifest.v2+json
    106 Media Type:  application/vnd.docker.distribution.manifest.v2+json
      1 Media Type:  application/vnd.oci.image.manifest.v1+json
ppc64le
     11 Media Type:    application/vnd.docker.distribution.manifest.v2+json
    115 Media Type:  application/vnd.docker.distribution.manifest.v2+json
      1 Media Type:  application/vnd.oci.image.manifest.v1+json
s390x
     11 Media Type:    application/vnd.docker.distribution.manifest.v2+json
    101 Media Type:  application/vnd.docker.distribution.manifest.v2+json
      1 Media Type:  application/vnd.oci.image.manifest.v1+json
x86_64
     11 Media Type:    application/vnd.docker.distribution.manifest.v2+json
    129 Media Type:  application/vnd.docker.distribution.manifest.v2+json
      1 Media Type:  application/vnd.oci.image.manifest.v1+json