Bug 1790380

Summary: 403 Forbidden when normal user view project metrics [openshift-4.4]
Product: OpenShift Container Platform Reporter: Yadan Pei <yapei>
Component: Management ConsoleAssignee: Rastislav Wagner <rawagner>
Status: CLOSED ERRATA QA Contact: Yadan Pei <yapei>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.4CC: aos-bugs, bpeterse, jokerman, juzhao, spadgett, yapei
Target Milestone: ---   
Target Release: 4.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-04 11:24:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
403 Forbidden
none
403 errors when logged in as test user
none
404 errors when logged in as kube:admin none

Description Yadan Pei 2020-01-13 09:30:02 UTC
Description of problem:
normal user view project metrics on Home -> Project -> Dashboard, it reports No datapoints found and all GET requests return 403 Forbidden. The issue is not reproduced by cluster-admin user

Version-Release number of selected component (if applicable):
4.4.0-0.nightly-2020-01-12-221811

How reproducible:
Always

Steps to Reproduce:
1. normal user create a project and add application, make sure some pods are running 
$ oc get pods -n ui1-project1 | grep Running
perl-1-bpks9            1/1     Running     0          95m
php-659cf5c84b-qgqbk    1/1     Running     0          36m
ruby-8486cb7467-5thrp   1/1     Running     0          36m
2. Check project status at Home -> Projects -> Dashboard


Actual results:
2. metrics in Utilization all report No datapoints found, GET request returns 403 Forbidden
Request URL: https://<console_route>/api/prometheus/api/v1/query_range?start=1578903086.444&end=1578906686.444&step=60&query=sum%28pod%3Acontainer_fs_usage_bytes%3Asum%7Bcontainer%3D%22%22%2Cpod%21%3D%22%22%2Cnamespace%3D%27ui1-project2%27%7D%29+BY+%28namespace%29


Expected results:
2. normal user should have permission to view metrics

Additional info:

Comment 1 Yadan Pei 2020-01-13 09:31:03 UTC
Created attachment 1651795 [details]
403 Forbidden

Comment 3 David Taylor 2020-02-05 15:05:15 UTC
Created attachment 1657918 [details]
403 errors when logged in as test user

Comment 4 David Taylor 2020-02-05 15:06:18 UTC
Created attachment 1657919 [details]
404 errors when logged in as kube:admin

Comment 5 David Taylor 2020-02-05 15:15:27 UTC
I was able to reproduce this using a 4.3 cluster which is needed at this time as workaround to:

https://bugzilla.redhat.com/show_bug.cgi?id=1794885
Prometheus and Alertmanager services returning 403 errors, breaking console metrics

Logged in as kube:admin, Projects -> Project Details, Utilization dashboard card shows graphs/data
Logged in as test:test, Projects -> Project Details, Utilization dashboard card shows 'Not available' & 'No datapoints found.'

Logged in as kube:admin, I see only 2 404 errors (see attached)
Logged in as test:test, I see several 403 errors (see attached)
- Not sure if errors due to running 4.4 code on top of 4.3 cluster, or part of the root cause

Debugging the error I see: "Error: Prometheus URL is not available at http://0.0.0.0:9000/static/main-0a3c6a98c951...."

Agree that normal user should be able to access '/api/prometheus/api/v1/query_range', as Prometheus docs states: "It is presumed that untrusted users have access to the Prometheus HTTP endpoint and logs. They have access to all time series information contained in the database, plus a variety of operational/debugging information. 
It is also presumed that only trusted users have the ability to change the command line, configuration file, rule files and other aspects of the runtime environment of Prometheus and other components."

Comment 6 David Taylor 2020-02-05 15:45:51 UTC
Issue seems to be here: https://github.com/openshift/console/blob/master/frontend/public/actions/dashboards.ts#L100
When logged in as test:test, window.SERVER_FLAGS.prometheusTenancyBaseURL and window.SERVER_FLAGS.prometheusBaseURL are empty strings
When logged in as kube:admin, these window.SERVER_FLAGS are set
Notice they are being set in server/server.go.

Comment 7 Samuel Padgett 2020-02-05 19:00:02 UTC
I don't believe that the project dashboard is passing the namespace with the query, so we're not hitting the prometheus tenancy endpoint.

Comment 8 Samuel Padgett 2020-02-05 19:02:54 UTC
Note that metrics are entirely broken by bug 1794885, but there is an additional problem specific to the project dashboard for normal users.

Comment 9 Rastislav Wagner 2020-02-06 13:52:21 UTC
the namespace passing got lost in https://github.com/openshift/console/pull/3790

Comment 11 Yadan Pei 2020-02-07 05:47:24 UTC
Now normal user can view project metrics successfully, charts in Utilization are shown correctly.

Verified on 4.4.0-0.nightly-2020-02-06-230833

Comment 14 Yadan Pei 2020-04-07 09:08:33 UTC
Moving to VERIFIED and opened a new bug to track this different issue

Comment 16 errata-xmlrpc 2020-05-04 11:24:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581