Bug 2105325

Summary: [oc adm release] extraction of the installer against a manifestlisted payload referenced by tag leads to a bad release image reference
Product: OpenShift Container Platform Reporter: aleskandro <adistefa>
Component: ocAssignee: Prashanth Sundararaman <psundara>
oc sub component: oc QA Contact: aleskandro <adistefa>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: medium CC: adistefa, gpei, lwan, mfojtik, psundara, yunjiang
Version: 4.11Keywords: TestBlocker
Target Milestone: ---   
Target Release: 4.12.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-01-17 19:51:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2108617    

Description aleskandro 2022-07-08 14:40:46 UTC 
Description of problem:

Extraction of openshift-install from a manifest list-based payload referenced by tag leads to the installer not having a well-formed reference to the payload for installation.

Version-Release number of selected component (if applicable):

Client Version: 4.11.0-202207072008.p0.gf17f1aa.assembly.stream-f17f1aa

Cluster Version/Payload under testing: 4.11.0-0.nightly-multi-2022-07-06-150106

How reproducible: Always

Steps to Reproduce:
1. oc adm release extract --command='openshift-install' quay.io/openshift-release-dev/ocp-release:4.11.0-0.nightly-multi-2022-07-06-150106
2. ./openshift-install version

Actual results:
 W0708 13:51:22.976430     312 manifest.go:442] Chose linux/amd64 manifest from the manifest list.
 W0708 13:51:23.653683     312 manifest.go:442] Chose linux/amd64 manifest from the manifest list.

./openshift-install 4.11.0-0.nightly-multi-2022-07-06-150106
 built from commit b2e7be726e400022e71ef3b8bd01a2093e53bc5a 
 release image quay.io/openshift-release-dev/ocp-release <<<< Missing digest here
 release architecture amd64

The missing digest leads to installation failure.

Expected results:

The same as extracting the command providing a manifest list payload by digest:

oc adm release extract --command='openshift-install' quay.io/openshift-release-dev/ocp-release@sha256:d9d8646ac3b4a944642cf660cd342942740c798350ad92e8f2672ee6d59c1ad2
./openshift-install version

./openshift-install 4.11.0-0.nightly-multi-2022-07-06-150106
 built from commit b2e7be726e400022e71ef3b8bd01a2093e53bc5a
 release image quay.io/openshift-release-dev/ocp-release@sha256:d9d8646ac3b4a944642cf660cd342942740c798350ad92e8f2672ee6d59c1ad2 <<<<<<<---
 release architecture amd64


or, for homogeneous payloads referenced by tag:

oc adm release extract --command='openshift-install' registry.ci.openshift.org/ocp/release:4.11.0-0.nightly-2022-07-06-145812 
 ./openshift-install 4.11.0-0.nightly-2022-07-06-145812
 built from commit b2e7be726e400022e71ef3b8bd01a2093e53bc5a
 release image registry.ci.openshift.org/ocp/release@sha256:616c5fefa87d116dd2440c75d9832c462078d635ed155c8d6cd486dd09540184
 release architecture amd64
Comment 1 aleskandro 2022-07-14 09:26:01 UTC 
Pre-merge verification passed on the proposed PR at commit ebe648da979d10b8ac90b6bbf60e15cc79ea8798.

Release and image info to check the manifestlist.

aleskandro@mifune ~ % ./oc adm release info quay.io/openshift-release-dev/ocp-release:4.11.0-0.nightly-multi-2022-07-13-114500
Name:           4.11.0-0.nightly-multi-2022-07-13-114500
Digest:         sha256:89c8352a09c8cac7f7d8f90af429375e4372f62ef8857347533daeae8a3250a7
Created:        2022-07-13T11:47:18Z
OS/Arch:        linux/amd64
Manifests:      593
Metadata files: 1

Pull From: quay.io/openshift-release-dev/ocp-release@sha256:89c8352a09c8cac7f7d8f90af429375e4372f62ef8857347533daeae8a3250a7

Release Metadata:
  Version:  4.11.0-0.nightly-multi-2022-07-13-114500
  Upgrades: <none>
  Metadata:
    release.openshift.io/architecture: multi

Component Versions:
  kubernetes 1.24.0
  machine-os 411.86.202207090519-0 Red Hat Enterprise Linux CoreOS

Images:
  NAME                                           DIGEST
  agent-installer-api-server                     sha256:09f876c058563716cbd02a6f892567c40269222242065cd18efd8479d1bf3e1c
....

aleskandro@mifune ~ % ./oc image info quay.io/openshift-release-dev/ocp-release:4.11.0-0.nightly-multi-2022-07-13-114500
error: the image is a manifest list and contains multiple images - use --filter-by-os to select from:

  OS            DIGEST
  linux/amd64   sha256:46c282d4f8a5781e7b710e05450d66334e640fe194726d7059b3e2474d7cece8
  linux/ppc64le sha256:e12db78d0ea4f72d55f5b23a94006c30e84ea47bdc78cda16d6fcfb50b5539f0
  linux/s390x   sha256:976409de1b5e9a792070d93f0e1c2486703e76468efc326fef7e8a4183479d9b
  linux/arm64   sha256:49aa3fe10e29c4eac215d298a7bd826a7ea97211bfb61a2f209736009b649f23


Release info by digest.

aleskandro@mifune ~ % ./oc adm release info quay.io/openshift-release-dev/ocp-release@sha256:89c8352a09c8cac7f7d8f90af429375e4372f62ef8857347533daeae8a3250a7
Name:           4.11.0-0.nightly-multi-2022-07-13-114500
Digest:         sha256:89c8352a09c8cac7f7d8f90af429375e4372f62ef8857347533daeae8a3250a7
Created:        2022-07-13T11:47:18Z
OS/Arch:        linux/amd64
Manifests:      593
Metadata files: 1

Pull From: quay.io/openshift-release-dev/ocp-release@sha256:89c8352a09c8cac7f7d8f90af429375e4372f62ef8857347533daeae8a3250a7

Release Metadata:
  Version:  4.11.0-0.nightly-multi-2022-07-13-114500
  Upgrades: <none>
  Metadata:
    release.openshift.io/architecture: multi

Component Versions:
  kubernetes 1.24.0
  machine-os 411.86.202207090519-0 Red Hat Enterprise Linux CoreOS

Images:
  NAME                                           DIGEST
  agent-installer-api-server                     sha256:09f876c058563716cbd02a6f892567c40269222242065cd18efd8479d1bf3e1c
  ....

Extracting the installer from the same payload by digest and tag and checking that the same ManifestList release image is being used.

aleskandro@mifune ~ % ./oc adm release extract --command="openshift-install"  quay.io/openshift-release-dev/ocp-release@sha256:89c8352a09c8cac7f7d8f90af429375e4372f62ef8857347533daeae8a3250a7

aleskandro@mifune ~ % ./openshift-install version
./openshift-install 4.11.0-0.nightly-multi-2022-07-13-114500
built from commit cdb9627de7efb43ad7af53e7804ddd3434b0dc58
release image quay.io/openshift-release-dev/ocp-release@sha256:89c8352a09c8cac7f7d8f90af429375e4372f62ef8857347533daeae8a3250a7
release architecture amd64
aleskandro@mifune ~ % ./oc adm release extract --command="openshift-install"  quay.io/openshift-release-dev/ocp-release:4.11.0-0.nightly-multi-2022-07-13-114500

aleskandro@mifune ~ % ./openshift-install version
./openshift-install 4.11.0-0.nightly-multi-2022-07-13-114500
built from commit cdb9627de7efb43ad7af53e7804ddd3434b0dc58
release image quay.io/openshift-release-dev/ocp-release@sha256:89c8352a09c8cac7f7d8f90af429375e4372f62ef8857347533daeae8a3250a7
release architecture amd64

Regression on non ManifestListed payloads

aleskandro@mifune ~ % ./oc adm release extract --command="openshift-install"  oc adm release extract --tools quay.io/openshift-release-dev/ocp-release:4.10.22-aarch64

error: you may only specify a single image via --from or argument
aleskandro@mifune ~ % ./oc adm release extract --command="openshift-install"  quay.io/openshift-release-dev/ocp-release:4.10.22-aarch64

aleskandro@mifune ~ % ./openshift-install version
./openshift-install 4.10.22
built from commit d63a12ba0ec33d492093a8fc0e268a01a075f5da
release image quay.io/openshift-release-dev/ocp-release@sha256:d9b8e0dedfef8ee1c3af3463c85c5a080e71130b964e7bf7aca1049bf6946604
release architecture arm64
aleskandro@mifune ~ %
Comment 3 aleskandro 2022-07-19 09:52:44 UTC 
verified again with oc extracted from payload 4.12.0-0.nightly-multi-2022-07-19-092936 


aleskandro@mifune tmp (master*) % ./oc adm release extract quay.io/openshift-release-dev/ocp-release:4.11.0-0.nightly-multi-2022-07-19-022323 --command=openshift-install
aleskandro@mifune tmp (master*) % ./openshift-install version
./openshift-install 4.11.0-0.nightly-multi-2022-07-19-022323
built from commit 37684309bcb598757c99d3ea9fbc0758343d64a5
release image quay.io/openshift-release-dev/ocp-release@sha256:be6d00941c78f9f31ae5f8cffa9fdad4b344332b8ea4e965eba7a2803a4fe52a
release architecture amd64
aleskandro@mifune tmp (master*) % ./oc adm release extract quay.io/openshift-release-dev/ocp-release@sha256:be6d00941c78f9f31ae5f8cffa9fdad4b344332b8ea4e965eba7a2803a4fe52a --command=openshift-install
aleskandro@mifune tmp (master*) % ./openshift-install version
./openshift-install 4.11.0-0.nightly-multi-2022-07-19-022323
built from commit 37684309bcb598757c99d3ea9fbc0758343d64a5
release image quay.io/openshift-release-dev/ocp-release@sha256:be6d00941c78f9f31ae5f8cffa9fdad4b344332b8ea4e965eba7a2803a4fe52a
release architecture amd64
aleskandro@mifune tmp (master*) % ./oc adm release info quay.io/openshift-release-dev/ocp-release@sha256:be6d00941c78f9f31ae5f8cffa9fdad4b344332b8ea4e965eba7a2803a4fe52a
Name:           4.11.0-0.nightly-multi-2022-07-19-022323
Digest:         sha256:be6d00941c78f9f31ae5f8cffa9fdad4b344332b8ea4e965eba7a2803a4fe52a
Created:        2022-07-19T02:25:44Z
OS/Arch:        linux/amd64
Manifests:      593
Metadata files: 1

Pull From: quay.io/openshift-release-dev/ocp-release@sha256:be6d00941c78f9f31ae5f8cffa9fdad4b344332b8ea4e965eba7a2803a4fe52a

Release Metadata:
  Version:  4.11.0-0.nightly-multi-2022-07-19-022323
  Upgrades: <none>
  Metadata:
    release.openshift.io/architecture: multi

Component Versions:
  kubernetes 1.24.0                
  machine-os 411.86.202207150124-0 Red Hat Enterprise Linux CoreOS

Images:
  NAME                                           DIGEST
  agent-installer-api-server                     sha256:66e655cd8a3c95bde748afeda0b8bdbf58ee29e14f491f0e07bf890abbc502f3
...
aleskandro@mifune tmp (master*) % ./oc adm release info quay.io/openshift-release-dev/ocp-release:4.11.0-0.nightly-multi-2022-07-19-022323

Name:           4.11.0-0.nightly-multi-2022-07-19-022323
Digest:         sha256:be6d00941c78f9f31ae5f8cffa9fdad4b344332b8ea4e965eba7a2803a4fe52a
Created:        2022-07-19T02:25:44Z
OS/Arch:        linux/amd64
Manifests:      593
Metadata files: 1

Pull From: quay.io/openshift-release-dev/ocp-release@sha256:be6d00941c78f9f31ae5f8cffa9fdad4b344332b8ea4e965eba7a2803a4fe52a

Release Metadata:
  Version:  4.11.0-0.nightly-multi-2022-07-19-022323
  Upgrades: <none>
  Metadata:
    release.openshift.io/architecture: multi

Component Versions:
  kubernetes 1.24.0                
  machine-os 411.86.202207150124-0 Red Hat Enterprise Linux CoreOS

Images:
  NAME                                           DIGEST
  agent-installer-api-server                     sha256:66e655cd8a3c95bde748afeda0b8bdbf58ee29e14f491f0e07bf890abbc502f3
....
Comment 6 errata-xmlrpc 2023-01-17 19:51:48 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.12.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7399