Description of problem: admin user, login admin console, check "Control Plane status" in "Home -> Overview", API Request Success Rate Not available. debuged, prometheus API is: (1 - (sum(rate(apiserver_request_count{code=~"5.."}[5m])) or vector(0))/ sum(rate(apiserver_request_count[5m]))) * 100 apiserver_request_count is not exist for 4.5, 4.4 does have this metrics, please consider to use other metrics, or assign this bug to openshift-apiserver team *************************************************** # token=`oc sa get-token prometheus-k8s -n openshift-monitoring` # oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/label/__name__/values' | jq | grep "apiserver_request_count" no result these are metrics started with "apiserver_request" # oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://thanos-querier.openshift-monitoring.svc:9091/api/v1/label/__name__/values' | jq | grep "apiserver_request" % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 66982 0 66982 0 0 757k 0 --:--:-- --:--:-- --:--:-- 760k "apiserver_request:availability30d", "apiserver_request:burnrate1d", "apiserver_request:burnrate1h", "apiserver_request:burnrate2h", "apiserver_request:burnrate30m", "apiserver_request:burnrate3d", "apiserver_request:burnrate5m", "apiserver_request:burnrate6h", "apiserver_request_duration_seconds_bucket", "apiserver_request_duration_seconds_count", "apiserver_request_duration_seconds_sum", "apiserver_request_total", "cluster:apiserver_request_duration_seconds:mean5m", "cluster_quantile:apiserver_request_duration_seconds:histogram_quantile", "code:apiserver_request_total:increase30d", "code:apiserver_request_total:rate:sum", "code_resource:apiserver_request_total:rate5m", "code_verb:apiserver_request_total:increase30d", *************************************************** 4.4 *************************************************** $ token=`oc sa get-token prometheus-k8s -n openshift-monitoring` # oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/label/__name__/values' | jq | grep "apiserver_request_count" % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 57476 0 57476 0 0 1518k 0 --:--:-- --:--:-- --:--:-- 1559k "apiserver_request_count", *************************************************** Version-Release number of selected component (if applicable): 4.5.0-0.nightly-2020-05-10-180138 How reproducible: always Steps to Reproduce: 1. See the description 2. 3. Actual results: API Request Success Rate Not available Expected results: no issue Additional info:
Created attachment 1687659 [details] API Request Success Rate Not available
Reassigning to openshift-apiserver team since the metric is missing.
is apiserver_request_count replaced by apiserver_request_total? API Request Success Rate Prometheus expression is now: (1 - (sum(rate(apiserver_request_total{code=~"5.."}[5m])) or vector(0))/ sum(rate(apiserver_request_total[5m]))) * 100 "Control Plane status" in "Home -> Overview", API Request Success Rate now available
payload: 4.5.0-0.nightly-2020-05-13-202437
https://github.com/openshift/console/commit/03921d8af9668427ea0994fde1c88c77912bb6d1 Moving to console team.
API Request Success Rate Prometheus expression is now: (1 - (sum(rate(apiserver_request_total{code=~"5.."}[5m])) or vector(0))/ sum(rate(apiserver_request_total[5m]))) * 100 "Control Plane status" in "Home -> Overview", API Request Success Rate now available apiserver_request_count is replaced by apiserver_request_total payload: 4.5.0-0.nightly-2020-05-19-041951
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2409