Red Hat Bugzilla – Bug 1498549
Heapster API gives "Forbidden" error message to cluster-reader user/SA
Last modified: 2017-11-01 11:47:26 EDT
Description of problem: When querying the Heapster API with cluster-reader user, I get a 403 ("Forbidden") response: # oadm policy add-cluster-role-to-user cluster-reader # oc login -u admin # curl -H "Authorization: Bearer $(oc whoami -t)" -X GET -k $(oc whoami --show-server)/api/v1/proxy/namespaces/openshift-infra/services/https:heapster:/api/v1/model/metrics { "kind": "Status", "apiVersion": "v1", "metadata": {}, "status": "Failure", "message": "User \"admin\" cannot \"proxy\" \"services\" with name \"https:heapster:\" in project \"openshift-infra\"", "reason": "Forbidden", "details": { "name": "https:heapster:", "kind": "services" }, "code": 403 } # It works with the same user granted with cluster-admin role: # oc whoami system:admin # oadm policy add-cluster-role-to-user cluster-admin admin # oc login -u admin # curl -H "Authorization: Bearer $(oc whoami -t)" -X GET -k $(oc whoami --show-server)/api/v1/proxy/namespaces/openshift-infra/services/https:heapster:/api/v1/model/metrics [ "cpu/usage_rate", "memory/usage", "cpu/request", "cpu/limit", "memory/request", "memory/limit" ] Version-Release number of selected component (if applicable): OCP3.4 How reproducible: 100% Steps to Reproduce: Explained above Actual results: Heapster API cannot be queried by a cluster-reader user. Expected results: As stated in [0], the API should be accessible for either cluster-reader or cluster-admin privileges. [0] https://github.com/openshift/origin-metrics#accessing-heapster-directly Additional info:
Are you sure your user has cluster-reader privileges?
# oc whoami system:admin # oc describe clusterPolicyBindings :default | egrep 'RoleBinding\[cluster-admins\]|RoleBinding\[cluster-readers\]|RoleBinding\[rolebinding-reader\]' -A5 RoleBinding[cluster-admins]: Role: cluster-admin Users: system:admin Groups: system:cluster-admins ServiceAccounts: <none> Subjects: <none> RoleBinding[cluster-readers]: Role: cluster-reader Users: admin Groups: system:cluster-readers ServiceAccounts: management-infra/management-admin, openshift-infra/heapster, logging/aggregated-logging-fluentd Subjects: <none> # oc login -u admin ... # oc whoami admin # curl -H "Authorization: Bearer $(oc whoami -t)" -X GET -k $(oc whoami --show-server)/api/v1/proxy/namespaces/openshift-infra/services/https:heapster:/api/v1/model/metrics { "kind": "Status", "apiVersion": "v1", "metadata": {}, "status": "Failure", "message": "User \"admin\" cannot \"proxy\" \"services\" with name \"https:heapster:\" in project \"openshift-infra\"", "reason": "Forbidden", "details": { "name": "https:heapster:", "kind": "services" }, "code": 403 } #
Just pasted the confirmation in the previous message (I did it also in the Bugzilla description)
I can reproduce, getting the same error message: "User admin cannot proxy services with name https:heapster: in project openshift-infra" (reproduced in 3.6)
I've been able to fix that by editing the "admin" role (oc edit clusterroles admin) and adding: - apiGroups: [""] resources: ["services"] resourceNames: ["https:heapster:"] verbs: ["proxy"] Matt, I'm not sure how to build a generic solution from that.
@joel, I've tested the same with cluster-reader role (adding the api group for the "https:heapster" resourceName) and it works, I'm just wondering if there is any reason for no adding it directly to the cluster-reader role, instead of the admin.
If the role is missing the permission, lets reassign to the security team to see if they have any reasoning for making this change. Cluster reader is suppose to be able to read _everything_ (eg *), that is the point of the role.
@Nicolas, you're right I edited the admin role, but it's cluster-reader indeed which is expected to have this right.
Cluster reader must be guaranteed to only have read access to resources. Since that cannot be determined for the proxy verb / proxy sub resource, we cannot add those permissions to it. See bug 1486142
Also, you should not edit the built in cluster roles. Create your own role / binding to give the specific permissions you want to the specific subjects.
As per comment #9 and referenced bug we won't fix this one. Matt do you want to take back this bug? If not I will close it WONTFIX today.
I have updated the docs referenced in the description so that it no longer says that a cluster-reader user can access this endpoint: https://github.com/openshift/origin-metrics/pull/393 I think we can close this.