Description of problem: [*] In a cluster with image mirroring configured, when a pod is created with image as `image-registry.openshift-image-registry.svc:5000/openshift/cli:latest` (example), the pod fails due to ImagePullBackOff. - While analyzing the image-registry logs, it appears that somewhere while fetching the digest location by either search or cache, the digest fetched by image-registry pod is actually the `image-id` (as per podman inspect <image>). [*] We have confirmed the output of `oc adm release mirror..` command, and that reports a successful completion. - Some additional scenarios tested (referring digest as 'sha256:xyz'): Scenario 1: - When we specify the image as 'quay.io/<path>/<name>@sha256:xyz' within a pod, the pod comes up running as expected. Scenario 2: - When we specify the image as 'mirrored-registry/<path>/<name>@sha256:xyz' within the pod, it again comes up running as expected with correct digest. However, this mismatch of digest is observed only when we specify the path of OCP's internal registry (image-registry.openshift-image-registry.svc:5000) Version-Release number of selected component (if applicable): - OCP v4.7.8 Actual results: - Image pull fails. Expected results: - Image pulls should be successful. Additional info: - The CU have mirror registries configured in OCP v4.7 as well, but ever since upgrading to v4.8, they are encountering this issue.
Both digests are correct. The client gets an error when it tried to pull the blob sha256:84c9...5f. This is not an image digest, it's a blob digest (images consist of blobs). It gets an error because the image registry fails to proxy it from the mirror registry: level=error msg="response completed with error" err.code=unknown err.detail="Get \"https://<REDACTED>/blobs/sha256:84c9<REDACTED>5f\": unauthorized: Not Authorized." err.message="unknown error" This is because the registry's implementation of ICSP in 4.8 does not support authorization. The problem should be fixed in 4.9. *** This bug has been marked as a duplicate of bug 1972009 ***