Bug 1791001
| Summary: | Metric labels set incorrectly during build | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Pawel Krupa <pkrupa> | ||||||
| Component: | Service Catalog | Assignee: | Jesus M. Rodriguez <jesusr> | ||||||
| Status: | CLOSED WONTFIX | QA Contact: | Fan Jia <jfan> | ||||||
| Severity: | unspecified | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 4.4 | CC: | juzhao, scuppett | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | 4.4.0 | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2020-02-26 23:43:12 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Bug Depends On: | 1804988 | ||||||||
| Bug Blocks: | |||||||||
| Attachments: |
|
||||||||
*** Bug 1798305 has been marked as a duplicate of this bug. *** test env:
cv: 4.4.0-0.nightly-2020-02-21-011943
release log include this bug fix.
test result:
1. Start a cluster
2. Go to metrics console
3. search for `kubernetes_build_info{job!~"kube-dns|coredns",gitVersion!="v1.17.0"}`
Prometheus still returns metrics with invalid label values for svcat-apiserver & svcat-controller-manager:
`gitCommit="$Format:%H$" and gitVersion="v0.0.0-master+$Format:%h$"`
Created attachment 1664840 [details]
svcat build_info shows up correctly
Created attachment 1664841 [details]
but kubernetes_build_info isn't being populated correctly
it is true that we can find "openshift_cluster_svcat_controller_manager_operator_build_info" and "openshift_cluster_svcat_controller_manager_operator_build_info" from Prometheus, it is also true that the "kubernetes_build_info" also exposed by openshift-service-catalog-apiserver-operator and openshift-service-catalog-controller-manager-operator
****************************************************************************************
openshift_cluster_svcat_apiserver_operator_build_info
Element Value
openshift_cluster_svcat_apiserver_operator_build_info{endpoint="https",gitCommit="62f9868a7884211668c87587cf4fbeeea6f06bcb",gitVersion="62f9868a",instance="10.130.0.24:8443",job="metrics",namespace="openshift-service-catalog-apiserver-operator",pod="openshift-service-catalog-apiserver-operator-5565dbb697-55cds",service="metrics"} 1
openshift_cluster_svcat_controller_manager_operator_build_info
Element Value
openshift_cluster_svcat_controller_manager_operator_build_info{endpoint="https",gitCommit="217034e0985c602780d95c4abb5c807dfb587e1c",gitVersion="217034e0",instance="10.130.0.23:8443",job="metrics",namespace="openshift-service-catalog-controller-manager-operator",pod="openshift-service-catalog-controller-manager-operator-8496m22hs",service="metrics"} 1
****************************************************************************************
count(kubernetes_build_info{job!~"kube-dns|coredns",gitVersion="v0.0.0-master+$Format:%h$"}) by (gitCommit, gitVersion,namespace)
Element Value
{gitCommit="$Format:%H$",gitVersion="v0.0.0-master+$Format:%h$",namespace="openshift-service-catalog-apiserver-operator"} 1
{gitCommit="$Format:%H$",gitVersion="v0.0.0-master+$Format:%h$",namespace="openshift-service-catalog-controller-manager-operator"} 1
****************************************************************************************
# oc -n openshift-service-catalog-apiserver-operator get pod
NAME READY STATUS RESTARTS AGE
openshift-service-catalog-apiserver-operator-5565dbb697-55cds 1/1 Running 7 19h
# oc -n openshift-service-catalog-apiserver-operator get ep
NAME ENDPOINTS AGE
metrics 10.130.0.24:8443 19h
# token=`oc sa get-token prometheus-k8s -n openshift-monitoring`
# oc -n openshift-service-catalog-apiserver-operator exec openshift-service-catalog-apiserver-operator-5565dbb697-55cds -- curl -k -H "Authorization: Bearer $token" https://10.130.0.24:8443/metrics | grep "kubernetes_build_info"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 12959 0 12959 0 0 95805 0 --:--:-- --:--:-- --:--:-- 95992
# HELP kubernetes_build_info [ALPHA] A metric with a constant '1' value labeled by major, minor, git version, git commit, git tree state, build date, Go version, and compiler from which Kubernetes was built, and platform on which it is running.
# TYPE kubernetes_build_info gauge
kubernetes_build_info{buildDate="1970-01-01T00:00:00Z",compiler="gc",gitCommit="$Format:%H$",gitTreeState="",gitVersion="v0.0.0-master+$Format:%h$",goVersion="go1.13.4",major="",minor="",platform="linux/amd64"} 1
****************************************************************************************
# oc -n openshift-service-catalog-controller-manager-operator get pod
NAME READY STATUS RESTARTS AGE
openshift-service-catalog-controller-manager-operator-8496m22hs 1/1 Running 5 19h
# oc -n openshift-service-catalog-controller-manager-operator get ep
NAME ENDPOINTS AGE
metrics 10.130.0.23:8443 19h
# token=`oc sa get-token prometheus-k8s -n openshift-monitoring`
# oc -n openshift-service-catalog-controller-manager-operator exec openshift-service-catalog-controller-manager-operator-8496m22hs -- curl -k -H "Authorization: Bearer $token" https://10.130.0.23:8443/metrics | grep "kubernetes_build_info"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 12990 0 12990 0 0 38770 0 --:--:-- --:--:-- --:--:-- 38892
# HELP kubernetes_build_info [ALPHA] A metric with a constant '1' value labeled by major, minor, git version, git commit, git tree state, build date, Go version, and compiler from which Kubernetes was built, and platform on which it is running.
# TYPE kubernetes_build_info gauge
kubernetes_build_info{buildDate="1970-01-01T00:00:00Z",compiler="gc",gitCommit="$Format:%H$",gitTreeState="",gitVersion="v0.0.0-master+$Format:%h$",goVersion="go1.13.4",major="",minor="",platform="linux/amd64"} 1
Since we expose openshift_cluster_svcat_apiserver_operator_build_info and openshift_cluster_svcat_controller_manager_operator_build_info correctly, that is sufficient to allow customers and support to get what version is being used in a cluster. We are not going to fix the the kubernetes_build_info for the service catalog. |
Description of problem: Following metric labels are not set: `gitCommit` and `gitVersion`. They are returned as `gitCommit="$Format:%H$",gitVersion="v0.0.0-master+$Format:%h$"` This is the case for openshift-service-catalog-apiserver-operator and for openshift-service-catalog-controller-manager-operator Version-Release number of selected component (if applicable): 4.4.0 How reproducible: Always Steps to Reproduce: 1. Start a cluster 2. Go to metrics console 3. search for `kubernetes_build_info{job!~"kube-dns|coredns",gitVersion!="v1.17.0"}` Actual results: Prometheus returns metrics with invalid label values: `gitCommit="$Format:%H$" and gitVersion="v0.0.0-master+$Format:%h$"` Expected results: Nothing is returned Additional info: Frequently happens in CI builds.