Bug 1904297
| Summary: | The index image generated by "opm index prune" leaves unrelated images | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | checheng |
| Component: | OLM | Assignee: | Evan Cordell <ecordell> |
| OLM sub component: | OLM | QA Contact: | Jian Zhang <jiazha> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | medium | ||
| Priority: | medium | CC: | aos-bugs, dsover, ecordell, glennsa, jmalde, jokerman, krizza, mfojtik, mgreenbe, mvardhan, pneedle, sople |
| Version: | 4.6 | Keywords: | Triaged, UpcomingSprint |
| Target Milestone: | --- | ||
| Target Release: | 4.7.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: Stranded Operator image bundles that were not referenced by any channel entries remained after an `opm index prune` operation.
Consequence: This lead to unexpected index images being mirrored.
Fix: Stranded image bundles are now removed when an index is pruned.
Result: The unexpected images are not included when the Operator catalog is later mirrored.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-02-24 15:37:53 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1918012 | ||
Can you supply the output logs of the commands in question as well as the actual commands used? The resulting index from running `opm index prune` should only have a single package in it, so if it's still mirroring all of the operators in the index instead of just one, it means your `oc adm catalog mirror` command ran against an index that had more than one package. Can you confirm that the index image referenced in `oc adm catalog mirror` was the pruned index and not the public default index? @Kevin Rizza Thank you for responsing. > Can you supply the output logs of the commands in question as well as the actual commands used? I will ready the log for you. > The resulting index from running `opm index prune` should only have a single package in it, so if it's still mirroring all of the operators in the index instead of just one, it means your `oc adm catalog mirror` command ran against an index that had more than one package. Yes, I think the result of `opm index prune` is right. But `oc adm catalog mirror` command mirror all images. > Can you confirm that the index image referenced in `oc adm catalog mirror` was the pruned index and not the public default index? Yes, the customer and my teamer confirm it. `oc adm catalog mirror` will mirror all images. Best regards. With the information in https://github.com/operator-framework/operator-registry/issues/542, I was able to reproduce this by adding a bundle to the database that was no longer referenced by a channel entry. 1, Build the opm with the fixed PR.
[root@preserve-olm-env operator-registry]# make build
GOFLAGS="-mod=vendor" go build -tags "json1" -o bin/appregistry-server ./cmd/appregistry-server
GOFLAGS="-mod=vendor" go build -tags "json1" -o bin/configmap-server ./cmd/configmap-server
GOFLAGS="-mod=vendor" go build -tags "json1" -o bin/initializer ./cmd/initializer
GOFLAGS="-mod=vendor" go build -tags "json1" -o bin/registry-server ./cmd/registry-server
GOFLAGS="-mod=vendor" go build -ldflags "-X 'github.com/operator-framework/operator-registry/cmd/opm/version.gitCommit=064f85a' -X 'github.com/operator-framework/operator-registry/cmd/opm/version.opmVersion=v1.14.3-90-g064f85a' -X 'github.com/operator-framework/operator-registry/cmd/opm/version.buildDate=2021-01-21T06:31:53Z'" -tags "json1" -o bin/opm ./cmd/opm
[root@preserve-olm-env operator-registry]#
[root@preserve-olm-env operator-registry]# ./bin/opm version
Version: version.Version{OpmVersion:"v1.14.3-90-g064f85a", GitCommit:"064f85a", BuildDate:"2021-01-21T06:31:53Z", GoOs:"linux", GoArch:"amd64"}
2, Prune the registry.redhat.io/redhat/redhat-operator-index:v4.6 to keep cluster-logging only. And, push it.
[root@preserve-olm-env operator-registry]# ./bin/opm index prune -f registry.redhat.io/redhat/redhat-operator-index:v4.6 -p cluster-logging -t quay.io/olmqe/redhat-operator-index:v4.6-2
INFO[0000] pruning the index packages="[cluster-logging]"
INFO[0000] Pulling previous image registry.redhat.io/redhat/redhat-operator-index:v4.6 to get metadata packages="[cluster-logging]"
INFO[0000] running /usr/bin/podman pull registry.redhat.io/redhat/redhat-operator-index:v4.6 packages="[cluster-logging]"
...
INFO[0013] [podman build --format docker -f index.Dockerfile070313030 -t quay.io/olmqe/redhat-operator-index:v4.6-2 .] packages="[cluster-logging]"
[root@preserve-olm-env operator-registry]# podman push quay.io/olmqe/redhat-operator-index:v4.6-2
Getting image source signatures
3, Check the database. There is no related images, looks good.
[root@preserve-olm-env operator-registry]# podman create quay.io/olmqe/redhat-operator-index:v4.6-2
46ea17a69db19e57bd5e8e0c44126bbcac074c32ffa851c3f64f01b83bc275e2
[root@preserve-olm-env operator-registry]# podman cp 46ea17a69db19e57bd5e8e0c44126bbcac074c32ffa851c3f64f01b83bc275e2:/database/index.db logging.db
[root@preserve-olm-env operator-registry]# sqlite3 logging.db "select distinct(operatorbundle_name) from related_image where operatorbundle_name not in (select operatorbundle_name from channel_entry) order by 1;"
4, check the image mirroring mapping.
[root@preserve-olm-env operator-registry]# oc adm catalog mirror quay.io/olmqe/redhat-operator-index:v4.6-2 localhost:5000 --manifests-only
src image has index label for database path: /database/index.db
using database path mapping: /database/index.db:/tmp/354694744
wrote database to /tmp/354694744
using database at: /tmp/354694744/index.db
no digest mapping available for quay.io/olmqe/redhat-operator-index:v4.6-2, skip writing to ImageContentSourcePolicy
wrote mirroring manifests to manifests-redhat-operator-index-1611212950
[root@preserve-olm-env operator-registry]# tree manifests-redhat-operator-index-1611212950/catalogSource.yaml
manifests-redhat-operator-index-1611212950/catalogSource.yaml [error opening dir]
0 directories, 0 files
[root@preserve-olm-env operator-registry]# tree manifests-redhat-operator-index-1611212950
manifests-redhat-operator-index-1611212950
├── catalogSource.yaml
├── imageContentSourcePolicy.yaml
└── mapping.txt
0 directories, 3 files
[root@preserve-olm-env operator-registry]# cat manifests-redhat-operator-index-1611212950/catalogSource.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: redhat-operator-index
namespace: openshift-marketplace
spec:
image: localhost:5000/olmqe/redhat-operator-index:v4.6-2
sourceType: grpc
[root@preserve-olm-env operator-registry]#
[root@preserve-olm-env operator-registry]# cat manifests-redhat-operator-index-1611212950/imageContentSourcePolicy.yaml
apiVersion: operator.openshift.io/v1alpha1
kind: ImageContentSourcePolicy
metadata:
name: redhat-operator-index
spec:
repositoryDigestMirrors:
- mirrors:
- localhost:5000/openshift4/ose-logging-curator5
source: registry.redhat.io/openshift4/ose-logging-curator5
- mirrors:
- localhost:5000/openshift4/ose-logging-fluentd
source: registry.redhat.io/openshift4/ose-logging-fluentd
- mirrors:
- localhost:5000/openshift4/ose-cluster-logging-operator-bundle
source: registry.redhat.io/openshift4/ose-cluster-logging-operator-bundle
- mirrors:
- localhost:5000/openshift4/ose-cluster-logging-operator
source: registry.redhat.io/openshift4/ose-cluster-logging-operator
[root@preserve-olm-env operator-registry]# cat manifests-redhat-operator-index-1611212950/mapping.txt
registry.redhat.io/openshift4/ose-logging-curator5@sha256:73884604ac4506bcfb2a3c112eb621f40e0cd53fede8118e0f7a1b292ac8f924=localhost:5000/openshift4/ose-logging-curator5:f6789123
registry.redhat.io/openshift4/ose-logging-fluentd@sha256:11ccb42f3d96b065f7d94879611a7aefabbe509b522c11ac36be7a1c959a34d6=localhost:5000/openshift4/ose-logging-fluentd:1f0b20f7
registry.redhat.io/openshift4/ose-cluster-logging-operator-bundle@sha256:61aca61840dcf1d50f4a17fc9b2e10b7855c563bd6680b4dc77e3e9283c81369=localhost:5000/openshift4/ose-cluster-logging-operator-bundle:8f6d6578
registry.redhat.io/openshift4/ose-cluster-logging-operator@sha256:540b0d087c5e5529bab555030310478630249a9339a4a4c3fac6d0d7037d5eac=localhost:5000/openshift4/ose-cluster-logging-operator:5f9e513e
quay.io/olmqe/redhat-operator-index:v4.6-2=localhost:5000/olmqe/redhat-operator-index:v4.6-2
Only "cluster-logging" related images list, looks good to me, verify it.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633 |
Description of problem: The need image index is limit at the `opm index prune` step. But `oc adm catalog mirror` ignore index image push all images into image mirror. When run Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. Get operator indexes of redhat-operator. $ podman run -p50051:50051 -it registry.redhat.io/redhat/redhat-operator-index:v4.6 2. Run the following command to prune the source index of all but the specified packages: $ opm index prune \ -f registry.redhat.io/redhat/redhat-operator-index:v4.6 \ -p cluster-logging \ -t <target_registry>:<port>/<namespace>/redhat-operator-index:v4.6 3. Run the following command to push the new index image to your target registry: $ podman push <target_registry>:<port>/<namespace>/redhat-operator-index:v4.6 4. Running oc adm catalog mirror command extracts the contents of an index image to generate the manifests required for mirroring. But all indexes is pushed that the index image is ignored. $ oc adm catalog mirror \ <index_image> \ <mirror_registry>:<port> \ -a ${REG_CREDS} \ --filter-by-os="<os>/<arch>" Actual results: Expected results: Additional info: The reference document https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html/operators/administrator-tasks#olm-restricted-networks