Bug 2060605
| Summary: | Broken access to public images: Unable to connect to the server: no basic auth credentials | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | W. Trevor King <wking> |
| Component: | Image Registry | Assignee: | Oleg Bulatov <obulatov> |
| Status: | CLOSED ERRATA | QA Contact: | Mike Fiedler <mifiedle> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 4.10 | CC: | aos-bugs, deads, mifiedle, sdodson, stbenjam, wewang, xiuwang |
| Target Milestone: | --- | Keywords: | FastFix, Regression |
| Target Release: | 4.11.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: the image registry announced that it supports basic auth
Consequence: when credentials are not provided, the client refuses to interact with the registry
Fix: remove announcement of basic auth
Result: clients can pull public images anonymously
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-08-10 10:52:11 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 2060610 | ||
|
Description
W. Trevor King
2022-03-03 20:27:58 UTC
(In reply to W. Trevor King from comment #0) > $ oc adm policy add-role-to-group system:image-puller system:authenticated --namespace openshift This grant is irrelevant for our unauthenticated test, and I get the same '' after rolling it back: $ oc adm policy remove-role-from-group system:image-puller system:unauthenticated --namespace openshift clusterrole.rbac.authorization.k8s.io/system:image-puller removed: "system:unauthenticated" $ oc image info --insecure "${HOST}/openshift/tools" Unable to connect to the server: no basic auth credentials So simple reproducer is: oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge oc adm policy add-role-to-group system:image-puller system:unauthenticated --namespace openshift sleep 4 # not sure, must need some time to let the registry operator create the Route HOST=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}') oc image info --insecure "${HOST}/openshift/tools" And you should see a successful fetch, but instead 4.10.0 gives 'no basic auth credentials' Marking blocker+ for 4.10.0. This bug prevents anonymous image pulls from the internal openshift image-registry. 1. This is a regression from 4.9 behavior. 2. There is no workaround. 3. We have no way of knowing if clusters are impacted before they upgrade 4. image-registry pulls are critical path on containerized workloads. We don't have data on frequency of anonymous pulls, but we have no reason to believe it is rare. Many (most?) images are not sensitive. I use `echo '{}' > no-auth.json && oc image info -a no-auth.json --insecure "${HOST}/openshift/tools:latest"` as the final command to confirm that anonymous image pulls function. On 4.10.rc.7, that command fails with
> [deads@fedora openshift]$ echo '{}' > no-auth.json && oc image info -a no-auth.json --insecure "${HOST}/openshift/tools:latest"
error: unable to read image default-route-openshift-image-registry.apps.ci-ln-h6ycc6t-72292.origin-ci-int-gce.dev.rhcloud.com/openshift/tools:latest: Head "https://default-route-openshift-image-registry.apps.ci-ln-h6ycc6t-72292.origin-ci-int-gce.dev.rhcloud.com/v2/openshift/tools/manifests/latest": no basic auth credentials
with the fix from #315 I see
> [deads@fedora openshift]$ echo '{}' > no-auth.json && oc image info -a no-auth.json --insecure "${HOST}/openshift/tools:latest"
Name: default-route-openshift-image-registry.apps.ci-ln-dmrzjkb-72292.origin-ci-int-gce.dev.rhcloud.com/openshift/tools:latest
Digest: sha256:3b5588eec3e985d73f246bc80dc618cff889da5ee652863fe750851e3e26dc09
Media Type: application/vnd.docker.distribution.manifest.v2+json
Created: 15d ago
Image Size: 273.7MB in 7 layers
Layers: 83.3MB sha256:eac1b95df832dc9f172fd1f07e7cb50c1929b118a4249ddd02c6318a677b506a
1.789kB sha256:47aa3ed2034c4f27622b989b26c06087de17067268a19a1b3642a7e2686cd1a3
16.81MB sha256:6a15bb59e41f721e4315ae053d006cfcdcf06bff2bd7586dacff36bf0272c30a
11.6MB sha256:996635267df8ca4c548dc4d34d06d19c54e9fc37dfc703a1058c2aa2e95bfd06
485.5kB sha256:81b704cb35c4fdab7096f5284395de67f8b9d163f69777c37e3e40a86aeaf238
48.77MB sha256:2edd04531b637ddb36e15719002efb85c91edbc27b0bbd894bd97311ed47979d
Verified with reproducer in comment 1 oc adm info did a successful fetch oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge oc adm policy add-role-to-group system:image-puller system:unauthenticated --namespace openshift sleep 4 # not sure, must need some time to let the registry operator create the Route HOST=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}') oc image info --insecure "${HOST}/openshift/tools" config.imageregistry.operator.openshift.io/cluster patched (no change) Warning: Group 'system:unauthenticated' not found clusterrole.rbac.authorization.k8s.io/system:image-puller added: "system:unauthenticated" W0303 16:02:52.613067 44202 helpers.go:151] Defaulting of registry auth file to "${HOME}/.docker/config.json" is deprecated. The default will be switched to podman config locations in the future version. Name: default-route-openshift-image-registry.apps.ci-ln-dmrzjkb-72292.origin-ci-int-gce.dev.rhcloud.com/openshift/tools:latest Digest: sha256:3b5588eec3e985d73f246bc80dc618cff889da5ee652863fe750851e3e26dc09 Media Type: application/vnd.docker.distribution.manifest.v2+json Created: 15d ago Image Size: 273.7MB in 7 layers Layers: 83.3MB sha256:eac1b95df832dc9f172fd1f07e7cb50c1929b118a4249ddd02c6318a677b506a 1.789kB sha256:47aa3ed2034c4f27622b989b26c06087de17067268a19a1b3642a7e2686cd1a3 16.81MB sha256:6a15bb59e41f721e4315ae053d006cfcdcf06bff2bd7586dacff36bf0272c30a 11.6MB sha256:996635267df8ca4c548dc4d34d06d19c54e9fc37dfc703a1058c2aa2e95bfd06 485.5kB sha256:81b704cb35c4fdab7096f5284395de67f8b9d163f69777c37e3e40a86aeaf238 48.77MB sha256:2edd04531b637ddb36e15719002efb85c91edbc27b0bbd894bd97311ed47979d 112.8MB sha256:7d1c2db5ae0d6e6acbd2027066bc4e17a6e41d1cc0e4831b8a167ca05ef53ce2 OS: linux Arch: amd64 Command: /usr/bin/bash <snip - rest of output removed> *** Bug 2017682 has been marked as a duplicate of this bug. *** Verified on 4.11.0-0.nightly-2022-03-06-112819 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5069 |