Using "--filter-by-os=.*" is not the solution to your problem, since that will make ALL architecture to be mirrored. My guess is that the two missing images mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1879877#c0 (namely Elasticsearch operator and the Logging operator) might be missing the necessary tags denoting their OS. From our tests, the command works as expected and the linked PR is unrelated to the described problem. I'm passing this over to the OLM team to double check.
This isn't possible today with OLM operators. OLM relies on manifestlist multiarch images in order to allow multiarch workflows, even when a particular cluster only runs on a particular architecture. The manifests of these individual operators have hardcoded references to specific image shas (that point to the manifestlist images, not the underlying arch specific images). The problem is that today all implementations of the docker v2 registry spec require that, in order to pull a manifestlist image, all of the underlying manifests in that image need to exist on the registry as well. Because oc adm catalog mirror has no way to rebuild these manifestlist images to only include the arch specified, --filter-by-os=.* is required otherwise any operator that actually supports multiple architectures will be broken. I'm closing this ticket as NOTABUG. There is no way for oc adm catalog mirror to implement this filtering without a new image registry spec, which is outside the control of this product. There is a related docs bug referenced above that is updating the oc adm catalog mirror documentation to require --filter-by-os=.*, which should get any customer unblocked.
*** Bug 1910823 has been marked as a duplicate of this bug. ***
*** Bug 1882689 has been marked as a duplicate of this bug. ***
Created attachment 1746223 [details] Error screen shots
*** Bug 1913225 has been marked as a duplicate of this bug. ***
*** Bug 1890951 has been marked as a duplicate of this bug. ***
Created attachment 1747319 [details] Error_Screenshot_02822706_1
[root@preserve-olm-env data]# oc version -o yaml clientVersion: buildDate: "2021-01-16T22:37:00Z" compiler: gc gitCommit: 4d52be6017616fe8836a51b820777df2f71d179f gitTreeState: clean gitVersion: 4.7.0-202101162121.p0-4d52be6 goVersion: go1.15.5 major: "" minor: "" platform: linux/amd64 releaseClientVersion: 4.7.0-0.nightly-2021-01-17-211555 ... 1, Specify a new flag '--index-filter-by-os=' to select the arch of the index image [root@preserve-olm-env data]# oc adm catalog mirror --help Mirrors the contents of a catalog into a registry. ... --filter-by-os='': Use --index-filter-by-os instead. A regular expression to control which index image is picked when multiple variants are available. Images will be passed as '<platform>/<architecture>[/<variant>]'. This does not apply to images referenced by the index. ... --index-filter-by-os='': A regular expression to control which index image is picked when multiple variants are available. Images will be passed as '<platform>/<architecture>[/<variant>]'. This does not apply to images referenced by the index. ... 2, it works well, looks good to me. [root@preserve-olm-env data]# oc adm catalog mirror --filter-by-os='.*' --index-filter-by-os='linux/amd64' registry.redhat.io/redhat/redhat-operator-index:v4.6 localhost:5000 --manifests-only src image has index label for database path: /database/index.db using database path mapping: /database/index.db:/tmp/579480556 W0119 02:40:22.580322 23620 manifest.go:440] Chose linux/amd64 manifest from the manifest list. wrote database to /tmp/579480556 using database at: /tmp/579480556/index.db no digest mapping available for registry.redhat.io/openshift-service-mesh/kiali-rhel7-operator:1.0.5, skip writing to ImageContentSourcePolicy ... no digest mapping available for registry.redhat.io/amq7/amq-online-1-mqtt-lwt:1.4, skip writing to ImageContentSourcePolicy wrote mirroring manifests to manifests-redhat-operator-index-1611024019 [root@preserve-olm-env data]# tree manifests-redhat-operator-index-1611024019 manifests-redhat-operator-index-1611024019 ├── catalogSource.yaml ├── imageContentSourcePolicy.yaml └── mapping.txt 0 directories, 3 files [root@preserve-olm-env data]# cat manifests-redhat-operator-index-1611024019/catalogSource.yaml apiVersion: operators.coreos.com/v1alpha1 kind: CatalogSource metadata: name: redhat-operator-index namespace: openshift-marketplace spec: image: localhost:5000/redhat/redhat-operator-index:v4.6 sourceType: grpc [root@preserve-olm-env data]# cat manifests-redhat-operator-index-1611024019/mapping.txt registry.redhat.io/rhacm2/search-api-rhel8@sha256:1a99fe01f0815106b08914e0697ced65f7a3d5a354ec8b121ed4fa7c76aa7ea4=localhost:5000/rhacm2/search-api-rhel8:8e475f23 registry.redhat.io/openshift-serverless-1-tech-preview/eventing-ping-source-rhel8@sha256:2064e979f1a624cb5016e9445a5532ec61da709c64b26f1548bed183dba3e30d=localhost:5000/openshift-serverless-1-tech-preview/eventing-ping-source-rhel8:519d787d ... Hi Sabarinath, One more question, as Kevin explained in comment 3, --filter-by-os='linux/amd64' is incorrect, --filter-by-os='.*' is required otherwise any operator that actually supports multiple architectures will be broken. But, why not you drop this `--filter-by-os` flag when doing `oc adm catalog mirror`? I guess it works well, as follows: [root@preserve-olm-env data]# ./oc adm catalog mirror registry.redhat.io/redhat/redhat-operator-index:v4.6 localhost:5000 --manifests-only ... Hi Kevin, As `--filter-by-os` doesn't work in selecting the specific platform/arch, can we remove this flag from `oc adm catalog mirror`? --filter-by-os='': Use --index-filter-by-os instead. A regular expression to control which index image is picked when multiple variants are available. Images will be passed as '<platform>/<architecture>[/<variant>]'. This does not apply to images referenced by the index. Change the status to ASSIGNED.
The patch that Evan created removes the need for --filter-by-os=.* to be explicitly set because it chooses it by default. However, we cannot just explicitly remove the flag without going through the deprecation process because of the rules around option removal in oc (not to mention that that is not something that could be backported to a previous release). I'm moving this back to ON_QA since it appears that the functionality itself is now working correctly. You are free to file an RFE to ask for that deprecation process to start, but I believe that is already the plan in a future release.
Note that the flag is deprecated in 4.7: https://github.com/openshift/oc/pull/710 (but that will not be backported to 4.6.z) To summarize: - `--filter-by-os` flag is still present, but only selects the variant of the index image itself (i.e. which arch of the registry.redhat.io/redhat/redhat-operator-index:v4.6 manifestlist is pulled and unpacked) - `--index-filter-by-os` does exactly the same thing, just with a different name to clarify that it does not affect content, only the initial index pull. - Everything will work as expected by completely omitting both flags! Don't worry about specifying arch anymore - We can't provide arch filtering for catalog content at this time.
Hi Kevin, Evan Thanks for your details! I will verify it once a new payload ready. Mark it as MODIFIED since no available payload contains this https://github.com/openshift/oc/pull/710
[root@preserve-olm-env data]# ./oc version -o yaml clientVersion: buildDate: "2021-01-21T00:15:43Z" compiler: gc gitCommit: 6f8f260853ad23a1edeb7ee622da764e6b711e37 gitTreeState: clean gitVersion: 4.7.0-202101202207.p0-6f8f260 goVersion: go1.15.5 major: "" minor: "" platform: linux/amd64 releaseClientVersion: 4.7.0-0.nightly-2021-01-21-012810 ... 1, --filter-by-os has been removed. [root@preserve-olm-env data]# ./oc adm catalog mirror --help Mirrors the contents of a catalog into a registry. ... --index-filter-by-os='': A regular expression to control which index image is picked when multiple variants are available. Images will be passed as '<platform>/<architecture>[/<variant>]'. This does not apply to images referenced by the index. 2, --index-filter-by-os works well. [root@preserve-olm-env data]# ./oc adm catalog mirror --index-filter-by-os='linux/amd64' registry.redhat.io/redhat/redhat-operator-index:v4.6 localhost:5000 --manifests-only src image has index label for database path: /database/index.db using database path mapping: /database/index.db:/tmp/982999206 W0121 02:12:59.201783 622 manifest.go:440] Chose linux/amd64 manifest from the manifest list. wrote database to /tmp/982999206 using database at: /tmp/982999206/index.db no digest mapping available for registry.redhat.io/container-native-virtualization/kubevirt-ssp-operator:v2.2.0-24, skip writing to ImageContentSourcePolicy ... wrote mirroring manifests to manifests-redhat-operator-index-1611195172 [root@preserve-olm-env data]# tree manifests-redhat-operator-index-1611195172 manifests-redhat-operator-index-1611195172 ├── catalogSource.yaml ├── imageContentSourcePolicy.yaml └── mapping.txt 0 directories, 3 files [root@preserve-olm-env data]# cat manifests-redhat-operator-index-1611195172/catalogSource.yaml apiVersion: operators.coreos.com/v1alpha1 kind: CatalogSource metadata: name: redhat-operator-index namespace: openshift-marketplace spec: image: localhost:5000/redhat/redhat-operator-index:v4.6 sourceType: grpc [root@preserve-olm-env data]# cat manifests-redhat-operator-index-1611195172/imageContentSourcePolicy.yaml apiVersion: operator.openshift.io/v1alpha1 kind: ImageContentSourcePolicy metadata: name: redhat-operator-index spec: repositoryDigestMirrors: - mirrors: - localhost:5000/openshift-serverless-1-tech-preview/serving-controller-rhel8 source: registry.redhat.io/openshift-serverless-1-tech-preview/serving-controller-rhel8 - mirrors: - localhost:5000/rhscl/postgresql-96-rhel7 source: registry.redhat.io/rhscl/postgresql-96-rhel7 ... [root@preserve-olm-env data]# cat manifests-redhat-operator-index-1611195172/mapping.txt registry.redhat.io/rhacm2/mcm-topology-rhel8@sha256:2b453e23ad8c6af0a2705f1e423aa5746574b6c3dca0d8130fdd60d076023d43=localhost:5000/rhacm2/mcm-topology-rhel8:1f21c843 registry.redhat.io/codeready-workspaces/pluginregistry-rhel8@sha256:e2f865550b46ead535c4b6cd8204889957e3bbc73300ad8af4e4e6a570249477=localhost:5000/codeready-workspaces/pluginregistry-rhel8:4673d8e1 ... Looks good to me, verify it.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days