Hide Forgot
Bug 1902456's [1] seems to have broken access to public images. Testing in a 4.10.0 cluster: $ oc get -o jsonpath='{.status.desired.version}{"\n"}' clusterversion version 4.10.0 Exposing the registry, per [2]: $ oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge $ HOST=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}') Grant public access to the openshift namespace (do we doc this somewhere): $ oc adm policy add-role-to-group system:image-puller system:unauthenticated --namespace openshift Warning: Group 'system:unauthenticated' not found clusterrole.rbac.authorization.k8s.io/system:image-puller added: "system:unauthenticated" $ oc adm policy add-role-to-group system:image-puller system:authenticated --namespace openshift Warning: Group 'system:authenticated' not found clusterrole.rbac.authorization.k8s.io/system:image-puller added: "system:authenticated" Try to fetch image metadata: $ oc image info --insecure "${HOST}/openshift/tools" Unable to connect to the server: no basic auth credentials For comparison, here is a cluster-bot cluster built from [3] (reverting [1]): $ oc get -o jsonpath='{.status.desired.version}{"\n"}' clusterversion version 4.10.0-0.ci.test-2022-03-03-194357-ci-ln-dmrzjkb-latest $ oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge $ HOST=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}') $ oc adm policy add-role-to-group system:image-puller system:unauthenticated --namespace openshift $ oc adm policy add-role-to-group system:image-puller system:authenticated --namespace openshift $ oc image info --insecure "${HOST}/openshift/tools:latest" Name: default-route-openshift-image-registry.apps.ci-ln-dmrzjkb-72292.origin-ci-int-gce.dev.rhcloud.com/openshift/tools:latest Digest: sha256:3b5588eec3e985d73f246bc80dc618cff889da5ee652863fe750851e3e26dc09 ... [1]: https://github.com/openshift/image-registry/pull/291 [2]: https://docs.openshift.com/container-platform/4.9/registry/securing-exposing-registry.html#registry-exposing-default-registry-manually_securing-exposing-registry [3]: https://github.com/openshift/image-registry/pull/315
(In reply to W. Trevor King from comment #0) > $ oc adm policy add-role-to-group system:image-puller system:authenticated --namespace openshift This grant is irrelevant for our unauthenticated test, and I get the same '' after rolling it back: $ oc adm policy remove-role-from-group system:image-puller system:unauthenticated --namespace openshift clusterrole.rbac.authorization.k8s.io/system:image-puller removed: "system:unauthenticated" $ oc image info --insecure "${HOST}/openshift/tools" Unable to connect to the server: no basic auth credentials So simple reproducer is: oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge oc adm policy add-role-to-group system:image-puller system:unauthenticated --namespace openshift sleep 4 # not sure, must need some time to let the registry operator create the Route HOST=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}') oc image info --insecure "${HOST}/openshift/tools" And you should see a successful fetch, but instead 4.10.0 gives 'no basic auth credentials'
Marking blocker+ for 4.10.0. This bug prevents anonymous image pulls from the internal openshift image-registry. 1. This is a regression from 4.9 behavior. 2. There is no workaround. 3. We have no way of knowing if clusters are impacted before they upgrade 4. image-registry pulls are critical path on containerized workloads. We don't have data on frequency of anonymous pulls, but we have no reason to believe it is rare. Many (most?) images are not sensitive.
I use `echo '{}' > no-auth.json && oc image info -a no-auth.json --insecure "${HOST}/openshift/tools:latest"` as the final command to confirm that anonymous image pulls function. On 4.10.rc.7, that command fails with > [deads@fedora openshift]$ echo '{}' > no-auth.json && oc image info -a no-auth.json --insecure "${HOST}/openshift/tools:latest" error: unable to read image default-route-openshift-image-registry.apps.ci-ln-h6ycc6t-72292.origin-ci-int-gce.dev.rhcloud.com/openshift/tools:latest: Head "https://default-route-openshift-image-registry.apps.ci-ln-h6ycc6t-72292.origin-ci-int-gce.dev.rhcloud.com/v2/openshift/tools/manifests/latest": no basic auth credentials with the fix from #315 I see > [deads@fedora openshift]$ echo '{}' > no-auth.json && oc image info -a no-auth.json --insecure "${HOST}/openshift/tools:latest" Name: default-route-openshift-image-registry.apps.ci-ln-dmrzjkb-72292.origin-ci-int-gce.dev.rhcloud.com/openshift/tools:latest Digest: sha256:3b5588eec3e985d73f246bc80dc618cff889da5ee652863fe750851e3e26dc09 Media Type: application/vnd.docker.distribution.manifest.v2+json Created: 15d ago Image Size: 273.7MB in 7 layers Layers: 83.3MB sha256:eac1b95df832dc9f172fd1f07e7cb50c1929b118a4249ddd02c6318a677b506a 1.789kB sha256:47aa3ed2034c4f27622b989b26c06087de17067268a19a1b3642a7e2686cd1a3 16.81MB sha256:6a15bb59e41f721e4315ae053d006cfcdcf06bff2bd7586dacff36bf0272c30a 11.6MB sha256:996635267df8ca4c548dc4d34d06d19c54e9fc37dfc703a1058c2aa2e95bfd06 485.5kB sha256:81b704cb35c4fdab7096f5284395de67f8b9d163f69777c37e3e40a86aeaf238 48.77MB sha256:2edd04531b637ddb36e15719002efb85c91edbc27b0bbd894bd97311ed47979d
Verified with reproducer in comment 1 oc adm info did a successful fetch oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge oc adm policy add-role-to-group system:image-puller system:unauthenticated --namespace openshift sleep 4 # not sure, must need some time to let the registry operator create the Route HOST=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}') oc image info --insecure "${HOST}/openshift/tools" config.imageregistry.operator.openshift.io/cluster patched (no change) Warning: Group 'system:unauthenticated' not found clusterrole.rbac.authorization.k8s.io/system:image-puller added: "system:unauthenticated" W0303 16:02:52.613067 44202 helpers.go:151] Defaulting of registry auth file to "${HOME}/.docker/config.json" is deprecated. The default will be switched to podman config locations in the future version. Name: default-route-openshift-image-registry.apps.ci-ln-dmrzjkb-72292.origin-ci-int-gce.dev.rhcloud.com/openshift/tools:latest Digest: sha256:3b5588eec3e985d73f246bc80dc618cff889da5ee652863fe750851e3e26dc09 Media Type: application/vnd.docker.distribution.manifest.v2+json Created: 15d ago Image Size: 273.7MB in 7 layers Layers: 83.3MB sha256:eac1b95df832dc9f172fd1f07e7cb50c1929b118a4249ddd02c6318a677b506a 1.789kB sha256:47aa3ed2034c4f27622b989b26c06087de17067268a19a1b3642a7e2686cd1a3 16.81MB sha256:6a15bb59e41f721e4315ae053d006cfcdcf06bff2bd7586dacff36bf0272c30a 11.6MB sha256:996635267df8ca4c548dc4d34d06d19c54e9fc37dfc703a1058c2aa2e95bfd06 485.5kB sha256:81b704cb35c4fdab7096f5284395de67f8b9d163f69777c37e3e40a86aeaf238 48.77MB sha256:2edd04531b637ddb36e15719002efb85c91edbc27b0bbd894bd97311ed47979d 112.8MB sha256:7d1c2db5ae0d6e6acbd2027066bc4e17a6e41d1cc0e4831b8a167ca05ef53ce2 OS: linux Arch: amd64 Command: /usr/bin/bash <snip - rest of output removed>
Related to https://bugzilla.redhat.com/show_bug.cgi?id=2017682
*** Bug 2017682 has been marked as a duplicate of this bug. ***
Verified on 4.11.0-0.nightly-2022-03-06-112819
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5069