Bug 1971710
| Summary: | Authentication error while running oc adm catalog mirror -a ${REG_CREDS} | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Aditya Deshpande <adeshpan> |
| Component: | oc | Assignee: | Nobody <nobody> |
| oc sub component: | oc | QA Contact: | zhou ying <yinzhou> |
| Status: | CLOSED EOL | Docs Contact: | |
| Severity: | medium | ||
| Priority: | medium | CC: | aos-bugs, maszulik, mfojtik, rjaiswal |
| Version: | 4.7 | ||
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-12-12 12:20:34 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Hello Can you please work on it on priority , as customer go live is very close. Regards Ravi (In reply to rjaiswal from comment #1) > Hello > > Can you please work on it on priority , as customer go live is very close. I'll try to provide some input later this week. Hello Maciez, Can you please update me when bug fix will be available ? Regards Ravi I just tried running these commands and I haven't stumbled on any issues as described, the only one that I hit with 4.7 was error: unable to push registry.connect.redhat.com/crunchydata/crunchy-pgbackrest: failed to upload blob sha256:4ca545ee6d5db5c1170386eeb39b2ffe3bd46e5d4a73a9acbebc805f19607eb3: Post "https://quay.io/v2/soltysh/crunchydata-crunchy-pgbackrest/blobs/uploads/": http2: server sent GOAWAY and closed the connection; LastStreamID=1, ErrCode=ENHANCE_YOUR_CALM, debug="" but that was recently fixed and will be soon available see https://bugzilla.redhat.com/show_bug.cgi?id=1976284 for details. Can I get -v=8 output from the mirror operation so that I can verify what might be the problem? I'd also suggest using the latest version of 4.7 oc which is 4.7.17 Hello Maciez, Can you please update us in your findings. Regards Ravi Closing since OCP 4.7 is out of support, see https://access.redhat.com/support/policy/updates/openshift#dates. Feel free to re-open against newer version of OCP, if the problem still appears. |
Description of problem: `oc adm catalog mirror` command is failing with authentication error to registry.connect.redhat.com though the -a ${REG_CREDS} parameter is mentioned and pull secret has proper credentials for registry.connect.redhat.com. ~~~ # cat /root/pull-secret.json { "auths": { "vm251-38.xxxx.xxxx.redhat.com:5200": { "auth": "YWRtaW46cmxxxx" }, "cloud.openshift.com": { "auth": "b3BlbnNoaWZ0LXJlbGxxxx", "email": "xxx" }, "quay.io": { "auth": "b3BlbnNoaWZ0LXJlbGVhc2UtZGVxxx", "email": "xxx" }, "registry.connect.redhat.com": { "auth": "NTE5NTYzNDB8dWhjLTFEd1hLak8xamJtRDNmNxxxx", "email": "xxx" }, "registry.redhat.io": { "auth": "NTE5NTYzNDB8dWhxxxxx", "email": "xxx" } } } # REG_CREDS=/root/pull-secret.json # echo $REG_CREDS /root/pull-secret.json --------------------------------------------------- ./oc adm catalog mirror vm251-38.xxxxx.xxxx.redhat.com:5200/olm-mirror2/certified-operator-index:v4.7 vm251-38.xxxxx.xxxx.redhat.com:5200/olm-mirror2 -a ${REG_CREDS} --insecure --filter-by-os='.*' Flag --filter-by-os has been deprecated, use --index-filter-by-os instead src image has index label for database path: /database/index.db using database path mapping: /database/index.db:/tmp/823175194 wrote database to /tmp/823175194 using database at: /tmp/823175194/index.db vm251-38.xxxxx.xxxx.redhat.com:5200/ [..] error: unable to retrieve source image registry.connect.redhat.com/enterprisedb/cloud-native-postgresql manifest sha256:4ac5d2ae655403f7cb1cede3d4bab2adf1d996b839bc289b76c5acd3299f3552: unauthorized: Access to the requested resource is not authorized error: unable to retrieve source image registry.connect.redhat.com/enterprisedb/cloud-native-postgresql manifest sha256:98a2c55449d5bd9e9e0ae4a9b49c1962ea7db4cce14ec46137b6ec1812b6d590: unauthorized: Access to the requested resource is not authorized [..] info: Mirroring completed in 6m58.36s (9.423MB/s) error mirroring image: one or more errors occurred no digest mapping available for vm251-38.xxxxx.xxxx.redhat.com:5200/olm-mirror2/certified-operator-index:v4.7, skip writing to ImageContentSourcePolicy wrote mirroring manifests to manifests-certified-operator-index-1623681670 ~~~ Sometimes it is observed for quay.io in case of running oc adm catalog mirror command again. error: unable to retrieve source image quay.io/enterprisedb/cloud-native-postgresql manifest sha256:3916a2ebd11c0a926d06d6ebfbd2cf6795ab01af598106ab7cb499ea26949879: unauthorized: authentication required error: unable to retrieve source image quay.io/enterprisedb/cloud-native-postgresql manifest sha256:bceb13937a77e2a7ae11a6cf18efc5a3b4b9fda4ef37de40d8c7248a8ba5a094: unauthorized: authentication required On the same host, Manual username/password login and image pull is working. ~~~ # podman login -u "5195xxxx|uhc-xxxx" -p "eyJhbxxx" registry.connect.redhat.com Login Succeeded! # podman pull registry.connect.redhat.com/enterprisedb/cloud-native-postgresql@sha256:4ac5d2ae655403f7cb1cede3d4bab2adf1d996b839bc289b76c5acd3299f3552 Trying to pull registry.connect.redhat.com/enterprisedb/cloud-native-postgresql@sha256:4ac5d2ae655403f7cb1cede3d4bab2adf1d996b839bc289b76c5acd3299f3552... Getting image source signatures Copying blob 46d615b6c79a done Copying blob 1ef4936d3906 done Copying blob 6f529754aaae done Copying config c07033b847 done Writing manifest to image destination Storing signatures c07033b8473f8b517fb2aa014795f6c0d63000218592dd2befecdb25c487e4c4 After running above checks, the oc adm catalog mirror command is failing again with authentication error. ~~~ error: unable to retrieve source image registry.connect.redhat.com/enterprisedb/cloud-native-postgresql manifest sha256:98a2c55449d5bd9e9e0ae4a9b49c1962ea7db4cce14ec46137b6ec1812b6d590: unauthorized: Access to the requested resource is not authorized error: unable to retrieve source image registry.connect.redhat.com/enterprisedb/cloud-native-postgresql manifest sha256:4ac5d2ae655403f7cb1cede3d4bab2adf1d996b839bc289b76c5acd3299f3552: unauthorized: Access to the requested resource is not authorized phase 0: ~~~ opm index prune was successful as below. ~~~ ./opm index prune -f registry.redhat.io/redhat/certified-operator-index:v4.7 -p cloud-native-postgresql,redis-enterprise-operator-cert,crunchy-postgres-operator -t vm251-38.xxxxx.xxxx.redhat.com:5200/olm-mirror2/certified-operator-index:v4.7 INFO[0000] pruning the index packages="[cloud-native-postgresql redis-enterprise-operator-cert crunchy-postgres-operator]" INFO[0000] Pulling previous image registry.redhat.io/redhat/certified-operator-index:v4.7 to get metadata packages="[cloud-native-postgresql redis-enterprise-operator-cert crunchy-postgres-operator]" INFO[0000] running /usr/bin/podman pull registry.redhat.io/redhat/certified-operator-index:v4.7 packages="[cloud-native-postgresql redis-enterprise-operator-cert crunchy-postgres-operator]" INFO[0010] running /usr/bin/podman pull registry.redhat.io/redhat/certified-operator-index:v4.7 packages="[cloud-native-postgresql redis-enterprise-operator-cert crunchy-postgres-operator]" [..] INFO[0027] Generating dockerfile packages="[cloud-native-postgresql redis-enterprise-operator-cert crunchy-postgres-operator]" INFO[0027] writing dockerfile: index.Dockerfile425392604 packages="[cloud-native-postgresql redis-enterprise-operator-cert crunchy-postgres-operator]" INFO[0027] running podman build packages="[cloud-native-postgresql redis-enterprise-operator-cert crunchy-postgres-operator]" INFO[0027] [podman build --format docker -f index.Dockerfile425392604 -t vm251-38.gsslab.pnq2.redhat.com:5200/olm-mirror2/certified-operator-index:v4.7 .] packages="[cloud-native-postgresql redis-enterprise-operator-cert crunchy-postgres-operator]" ~~~ Version-Release number of selected component (if applicable): # oc version Client Version: 4.7.9 # opm version Version: version.Version{OpmVersion:"v1.15.4-6-ga97d366a", GitCommit:"a97d366a92d302ff2056fa2d19aa3e48b0fbc99c", BuildDate:"2021-04-25T08:45:51Z", GoOs:"linux", GoArch:"amd64"} How reproducible: Always Steps to Reproduce: Follow the documentation: https://docs.openshift.com/container-platform/4.7/operators/admin/olm-restricted-networks.html for certified operators cloud-native-postgresql,redis-enterprise-operator-cert,crunchy-postgres-operator. Actual results: Catalog mirror is not happening and it is giving error for authentication. Expected results: Catalog mirror should happen successfully without any authentication error. Additional info: