Bug 2061785
| Summary: | Image registry uses ICSPs only when source exactly matches image | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Oleg Bulatov <obulatov> |
| Component: | Image Registry | Assignee: | Oleg Bulatov <obulatov> |
| Status: | CLOSED ERRATA | QA Contact: | XiuJuan Wang <xiuwang> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 4.8 | CC: | aos-bugs, xiuwang |
| Target Milestone: | --- | ||
| Target Release: | 4.10.z | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 2014240 | Environment: | |
| Last Closed: | 2022-03-28 12:03:50 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 2014240 | ||
| Bug Blocks: | 2086864 | ||
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.10.6 bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:1026 |
Verified on 4.10.0-0.ci.test-2022-03-09-075704-ci-ln-1xv7lx2-latest spec: repositoryDigestMirrors: - mirrors: - ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/openshifttest source: quay.io/openshifttest - mirrors: - ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000 source: registry.redhat.io Senario 1: sources and mirrors only defined registry hostname oc tag registry.redhat.io/rhel8/mysql-80@sha256:144c42e7437f4024eee4ce556d6ead03b4ec7d0ba636a0f1ce0ace9ace49c545 mysql:latest --reference-policy=local $oc create deployment mysql-deployment --image=image-registry.openshift-image-registry.svc:5000/default/mysql:latest -- sleep 300 $oc get pods Senario 2: sources and mirrors only defined registry hostname and namespace $oc tag quay.io/openshifttest/skopeo@sha256:426196e376cf045012289d53fec986554241496ed7f38e347fc56505aa8ad322 skopeo:latest --reference-policy=local $oc create deployment test-deployment --image=image-registry.openshift-image-registry.svc:5000/default/skopeo:latest -- sleep 300 oc get pods spec: repositoryDigestMirrors: - mirrors: - ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/app source: registry1-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar/myapp - mirrors: - ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/app source: registry2-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar - mirrors: - ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000 source: registry2-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar - mirrors: - ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/registry-example-com source: registry4-wxj.apps.wxj225a.qe.devcluster.openshift.com - mirrors: - ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/mirrored-foo source: registry5-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo $for i in {1..5}; do oc run pod$i --image=registry$i-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8 -- sleep 300 ; done Since the 5 source repositories don't exist, so failed to pull image, but the pull path are correct. $for i in {1..5}; do oc describe pods pod$i | grep "Failed to pull image"; done Warning Failed 7m55s (x4 over 9m26s) kubelet Failed to pull image "registry1-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8": rpc error: code = Unknown desc = (Mirrors also failed: [ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/app@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8: reading manifest sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8 in ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/app: manifest unknown: manifest unknown]): registry1-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8: pinging container registry registry1-wxj.apps.wxj225a.qe.devcluster.openshift.com: Get "https://registry1-wxj.apps.wxj225a.qe.devcluster.openshift.com/v2/": Forbidden Warning Failed 7m4s (x2 over 7m16s) kubelet Failed to pull image "registry2-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8": rpc error: code = Unknown desc = (Mirrors also failed: [ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8: reading manifest sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8 in ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/myapp: manifest unknown: manifest unknown]): registry2-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8: pinging container registry registry2-wxj.apps.wxj225a.qe.devcluster.openshift.com: Get "https://registry2-wxj.apps.wxj225a.qe.devcluster.openshift.com/v2/": Forbidden Warning Failed 5m57s (x2 over 6m39s) kubelet Failed to pull image "registry2-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8": rpc error: code = Unknown desc = (Mirrors also failed: [ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8: reading manifest sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8 in ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/myapp: manifest unknown: manifest unknown] Warning Failed 4m32s (x4 over 6m4s) kubelet Failed to pull image "registry3-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8": rpc error: code = Unknown desc = pinging container registry registry3-wxj.apps.wxj225a.qe.devcluster.openshift.com: Get "https://registry3-wxj.apps.wxj225a.qe.devcluster.openshift.com/v2/": Forbidden Warning Failed 4m10s (x4 over 5m39s) kubelet Failed to pull image "registry4-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8": rpc error: code = Unknown desc = (Mirrors also failed: [ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/registry-example-com/foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8: reading manifest sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8 in ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/registry-example-com/foo/bar/myapp: manifest unknown: manifest unknown]): registry4-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8: pinging container registry registry4-wxj.apps.wxj225a.qe.devcluster.openshift.com: Get "https://registry4-wxj.apps.wxj225a.qe.devcluster.openshift.com/v2/": Forbidden Warning Failed 3m35s (x4 over 4m58s) kubelet Failed to pull image "registry5-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8": rpc error: code = Unknown desc = (Mirrors also failed: [ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/mirrored-foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8: reading manifest sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8 in ec2-3-145-100-29.us-east-2.compute.amazonaws.com:5000/mirrored-foo/bar/myapp: manifest unknown: manifest unknown]): registry5-wxj.apps.wxj225a.qe.devcluster.openshift.com/foo/bar/myapp@sha256:ef3e5d3241c55c6e9e85771e604ee06170755b02bd8c6f8b372f8ec77c4684f8: pinging container registry registry5-wxj.apps.wxj225a.qe.devcluster.openshift.com: Get "https://registry5-wxj.apps.wxj225a.qe.devcluster.openshift.com/v2/": Forbidden