Bug 1832968

Summary: oc adm catalog mirror does not mirror the index image itself
Product: OpenShift Container Platform Reporter: lgallett
Component: OLMAssignee: Evan Cordell <ecordell>
OLM sub component: OLM QA Contact: kuiwang
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: medium CC: andbartl, dcain, ecordell, jiazha, krizza, nhale, rdave
Version: 4.4Keywords: Reopened, Upstream
Target Milestone: ---   
Target Release: 4.7.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-02-24 15:12:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1838444, 1917547    

Description lgallett 2020-05-07 14:36:17 UTC
Description of problem:

When mirroring an index image using oc adm catalog mirror, the index image itself does not get mirrored.
During the release process of bundle images, staging and dev index images are used in QE testing. The images are behind vpn and cannot be accessed on OCP clusters used by QE so these need to be mirrored as well.

Version-Release number of selected component (if applicable):

oc 4.4+

How reproducible:

always


Steps to Reproduce:
oc adm catalog mirror registry.stage.redhat.io/redhat/redhat-operator-index@sha256:fba04293ef3a555b7d689eb59e81f61599680c51183abe4c5e914809b01457b7 https://localhost:5000/

Actual results:

registry.stage.redhat.io/redhat/redhat-operator-index@sha256:fba04293ef3a555b7d689eb59e81f61599680c51183abe4c5e914809b01457b7 is not present in the ICSP or the mapping.txt

Expected results:
registry.stage.redhat.io/redhat/redhat-operator-index@sha256:fba04293ef3a555b7d689eb59e81f61599680c51183abe4c5e914809b01457b7 is present in the ICSP and the mapping.txt

Additional info:

The work around is to mirror the index image separately using oc image mirror and adding it to the ICSP manually.

Comment 3 Kevin Rizza 2020-08-20 19:32:48 UTC
Closing this as in 4.6 and later users will use `opm` to build their catalog images manually, and in that case it makes sense to just target the registry you are mirroring to.

Comment 4 Evan Cordell 2020-09-18 12:57:36 UTC
Re-opening as I believe this is still a UX problem that needs to be addressed.

Regardless of how the index image is made, `oc adm catalog mirror` is used to mirror it. It's weird to have an extra step of `oc image mirror <index image>` in addition to the `oc adm catalog mirror` step.

Comment 10 kuiwang 2020-11-03 02:33:39 UTC
Hi Evan,

  Verify it with the following.
--
[root@preserve-olm-env 1832968]# oc version
Client Version: 4.7.0-0.nightly-2020-11-02-231230
Server Version: 4.7.0-0.nightly-2020-10-27-051128
Kubernetes Version: v1.19.0+e67f5dc

[root@preserve-olm-env 1832968]# oc adm catalog mirror --manifests-only  registry.stage.redhat.io/redhat/redhat-operator-index@sha256:fba04293ef3a555b7d689eb59e81f61599680c51183abe4c5e914809b01457b7  quay.io/kuiwang
src image has index label for database path: /database/index.db
using database path mapping: /database/index.db:/tmp/644103008
wrote database to /tmp/644103008
using database at: /tmp/644103008/index.db
no digest mapping available for registry-proxy-stage.engineering.redhat.com/rh-osbs-stage/e2e-e2e-test-rhel8-operator:8.0-350, skip writing to ImageContentSourcePolicy
no digest mapping available for registry-proxy.engineering.redhat.com/rh-osbs/amq7-amq-streams-rhel7-operator-metadata:1.2.0-1, skip writing to ImageContentSourcePolicy
no digest mapping available for registry-proxy.engineering.redhat.com/rh-osbs/amq7-amq-streams-rhel7-operator-metadata:1.1.0-1, skip writing to ImageContentSourcePolicy
no digest mapping available for registry-proxy.engineering.redhat.com/rh-osbs/openshift-service-mesh-kiali-operator-metadata:1.0-7, skip writing to ImageContentSourcePolicy
no digest mapping available for registry-proxy.engineering.redhat.com/rh-osbs/amq7-amq-streams-rhel7-operator-metadata:1.3.0-1, skip writing to ImageContentSourcePolicy
no digest mapping available for registry.stage.redhat.io/openshift-service-mesh/kiali-rhel7-operator:1.0.5, skip writing to ImageContentSourcePolicy
no digest mapping available for registry-proxy.engineering.redhat.com/rh-osbs/amq7-amq-streams-rhel7-operator-metadata:1.4.0-1, skip writing to ImageContentSourcePolicy
no digest mapping available for registry.access.redhat.com/amq7/amq-streams-cluster-operator:1.1.0, skip writing to ImageContentSourcePolicy
no digest mapping available for registry-proxy.engineering.redhat.com/rh-osbs/lgallett-bundle:v1.1-14, skip writing to ImageContentSourcePolicy
no digest mapping available for registry.access.redhat.com/amqstreams-1/amqstreams10-clusteroperator-openshift:1.0.0, skip writing to ImageContentSourcePolicy
no digest mapping available for registry-proxy.engineering.redhat.com/rh-osbs/amq7-amq-streams-rhel7-operator-metadata:1.0.0-1, skip writing to ImageContentSourcePolicy
no digest mapping available for registry.redhat.io/amq7/amq-streams-operator:1.2.0, skip writing to ImageContentSourcePolicy
no digest mapping available for registry.redhat.io/amq7/amq-streams-operator:1.3.0, skip writing to ImageContentSourcePolicy
wrote mirroring manifests to manifests-redhat-operator-index-1604367900
[root@preserve-olm-env 1832968]# cd manifests-redhat-operator-index-1604367900/

[root@preserve-olm-env manifests-redhat-operator-index-1604367900]# cat mapping.txt |grep redhat-operator-index
registry.stage.redhat.io/redhat/redhat-operator-index@sha256:fba04293ef3a555b7d689eb59e81f61599680c51183abe4c5e914809b01457b7=quay.io/kuiwang/redhat-redhat-operator-index:3909dae0



[root@preserve-olm-env manifests-redhat-operator-index-1604367900]# cat imageContentSourcePolicy.yaml |grep redhat-operator-index
  name: redhat-operator-index
    - quay.io/kuiwang/redhat-redhat-operator-index
    source: registry.stage.redhat.io/redhat/redhat-operator-index
[root@preserve-olm-env manifests-redhat-operator-index-1604367900]# cat catalogSource.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: redhat-operator-index
  namespace: openshift-marketplace
spec:
  image: quay.io/kuiwang/redhat-redhat-operator-index@sha256:fba04293ef3a555b7d689eb59e81f61599680c51183abe4c5e914809b01457b7
  sourceType: grpc

--

But I am confused on the example catalogSource.yaml
[root@preserve-olm-env manifests-redhat-operator-index-1604367900]# cat catalogSource.yaml 
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: redhat-operator-index
  namespace: openshift-marketplace
spec:
  image: quay.io/kuiwang/redhat-redhat-operator-index@sha256:fba04293ef3a555b7d689eb59e81f61599680c51183abe4c5e914809b01457b7
  sourceType: grpc

Why does it take SHA(fba04293ef3a555b7d689eb59e81f61599680c51183abe4c5e914809b01457b7) which could be changed possible when mirroring it among different platform? and could we use the "quay.io/kuiwang/redhat-redhat-operator-index:3909dae0" which is mirrored?

I change it back to assigned and after we clarify it, I will change it to verified. Thanks.

Comment 11 Dave Cain 2020-11-10 13:00:35 UTC
The local-storage-operator is also broken in 4.6 disconnected (air-gapped) installs.

Comment 12 Evan Cordell 2020-11-10 15:00:20 UTC
> Why does it take SHA(fba04293ef3a555b7d689eb59e81f61599680c51183abe4c5e914809b01457b7) which could be changed possible when mirroring it among different platform? and could we use the "quay.io/kuiwang/redhat-redhat-operator-index:3909dae0" which is mirrored?

The tag is just added so that the image does not get GCd by the registry. I think the sha is preferable so that if you mirror the image again, an ICSP can be used without changing the ICSP. But I don't think it makes much of a difference from the user perspective.

The sha should not change when mirroring to a specific platform - the mirroring should keep the manifest list so that it's the same for all arches. If a user wanted to only mirror one arch, they could do so and simple update the CatalogSource to point to it.

Comment 14 kuiwang 2020-11-11 01:00:18 UTC
based on my verification and the information from Evan. LGTM

Comment 19 errata-xmlrpc 2021-02-24 15:12:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633