Description of problem: Pods referring to the internal openshift registry through an imagestream tag do not work when the referred image in the external registry has a dot in the name of its repo. Version-Release number of selected component (if applicable): How reproducible: 100% Steps to Reproduce: 1. Create an imagestream as below: oc apply -f - <<EOF apiVersion: image.openshift.io/v1 kind: ImageStream metadata: name: bb namespace: test-pat5 spec: lookupPolicy: local: true tags: - from: kind: DockerImage name: example.com/busy.box:v5 importPolicy: {} name: "v5a" referencePolicy: type: Local EOF 2. Deploy a pod that uses the imagestream tag name # oc run p --restart=Never --image=bb:v5a --image-pull-policy=Always -- echo "Hello" 3. Check if the pod has successfully pulled the image or not. # oc get events 13s Warning Failed pod/p Failed to pull image "image-registry.openshift-image-registry.svc:5000/test-pat5/bb@sha256:dd97a3fe6d721c5cf03abac0f50e2848dc583f7c4e41bf39102ceb42edfd1808": rpc error: code = Unknown desc = Error reading manifest sha256:dd97a3fe6d721c5cf03abac0f50e2848dc583f7c4e41bf39102ceb42edfd1808 in image-registry.openshift-image-registry.svc:5000/test-pat5/bb: unknown: unable to pull manifest from example.com/busy.box@sha256:dd97a3fe6d721c5cf03abac0f50e2848dc583f7c4e41bf39102ceb42edfd1808: invalid reference format Actual results: The image pull fails with the error "invalid reference format" Expected results: The image should be pulled successfully Additional info: https://github.com/heyvito/podman-macos/issues/8
Checked on 4.11.0-0.nightly-2022-02-11-014337 cluster with oc client build from https://github.com/dmage/oc/tree/namespace-with-dot $oc version Client Version: v4.2.0-alpha.0-1379-g3e7d761 Server Version: 4.11.0-0.nightly-2022-02-11-014337 Kubernetes Version: v1.23.3+f14faf2 still could reproduce this issue. Could mirror image with dot in namespace, but imagestream import still failed. $oc image mirror quay.io/openshifttest/busybox@sha256:c5439d7db88ab5423999530349d327b04279ad3161d7596d2126dfb5b02bfd1f registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.abcd:latest --keep-manifest-list=true --filter-by-os='.*' --insecure sha256:c5439d7db88ab5423999530349d327b04279ad3161d7596d2126dfb5b02bfd1f registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.abcd:latest info: Mirroring completed in 13.44s (776.8kB/s) $oc import-image wxj:v1 --from=registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.abcd:latest --insecure --confirm error: tag latest failed: Internal error occurred: registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.abcd1:latest: invalid reference format imagestream.image.openshift.io/wxj imported with errors Name: wxj Namespace: default Created: Less than a second ago Labels: <none> Annotations: openshift.io/image.dockerRepositoryCheck=2022-02-11T03:50:33Z Image Repository: image-registry.openshift-image-registry.svc:5000/default/wxj Image Lookup: local=false Unique Images: 0 Tags: 1 v1 tagged from registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.abcd1:latest will use insecure HTTPS or HTTP connections ! error: Import failed (InternalError): Internal error occurred: registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.abcd:latest: invalid reference format Less than a second ago
4.11.0-0.nightly-2022-02-11-014337 doesn't have the fix, 4.11.0-0.nightly-2022-02-11-053650 is the first build with the fix. Moving back to POST.
Thanks Oleg, I didn't notice https://github.com/openshift/openshift-apiserver/pull/279 not included in 4.11.0-0.nightly-2022-02-11-014337. Verified on 4.11.0-0.nightly-2022-02-11-053650 $oc version Client Version: v4.2.0-alpha.0-1379-g3e7d761 Server Version: 4.11.0-0.nightly-2022-02-11-053650 $oc image mirror quay.io/openshifttest/busybox@sha256:c5439d7db88ab5423999530349d327b04279ad3161d7596d2126dfb5b02bfd1f registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.myimage:latest --keep-manifest-list=true --filter-by-os='.*' --insecure W0211 19:02:41.261565 2782 helpers.go:151] Defaulting of registry auth file to "${HOME}/.docker/config.json" is deprecated. The default will be switched to podman config locations in the future version. registry-default.apps.wxjbz211.qe.devcluster.openshift.com/ test.myimage blobs: sha256:c5439d7db88ab5423999530349d327b04279ad3161d7596d2126dfb5b02bfd1f registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.myimage:latest info: Mirroring completed in 12.33s (846.6kB/s) $oc tag registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.myimage:latest bb1:v5a --reference-policy=local --insecure $oc import-image mytest:latest --from=registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.myimage:latest --insecure --confirm --reference-policy=local imagestream.image.openshift.io/mytest imported Name: mytest Namespace: wxj Created: Less than a second ago Labels: <none> Annotations: openshift.io/image.dockerRepositoryCheck=2022-02-11T11:13:20Z Image Repository: image-registry.openshift-image-registry.svc:5000/wxj/mytest Image Lookup: local=false Unique Images: 1 Tags: 1 latest tagged from registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.myimage:latest will use insecure HTTPS or HTTP connections prefer registry pullthrough when referencing this tag * registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.myimage@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019 Less than a second ago Image Name: mytest:latest Docker Image: registry-default.apps.wxjbz211.qe.devcluster.openshift.com/test.myimage@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019 Name: sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019 Created: Less than a second ago Annotations: image.openshift.io/dockerLayersOrder=ascending Image Size: 766.1kB in 1 layers Layers: 764.7kB sha256:e5d9363303ddee1686b203170d78283404e46a742d4c62ac251aae5acbda8df8 Image Created: 13 months ago Author: <none> Arch: amd64 Command: sh Working Dir: <none> User: <none> Exposes Ports: <none> Docker Labels: <none> Environment: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin $oc set image-lookup mytest $oc run mypod --restart=Never --image=mytest:latest --image-pull-policy=Always -- echo "Hello" pod/mypod created $oc describe pods mypod Pulling image "image-registry.openshift-image-registry.svc:5000/wxj/mytest@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019" Normal Pulled 5s kubelet Successfully pulled image "image-registry.openshift-image-registry.svc:5000/wxj/mytest@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019" in 673.969493ms
Verified on 4.11.0-0.nightly-2022-02-11-215438 cluster ./oc version Client Version: 4.11.0-0.nightly-2022-02-11-215438 Server Version: 4.11.0-0.nightly-2022-02-11-215438 Kubernetes Version: v1.23.3+f14faf2
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5069