Bug 1504034

Created attachment 1340681 [details]
missing uid label on OCP-3.7 Prometheus

Description of problem:

OCP 3.7 will use Prometheus metrics. It is impossible to query metrics using the uid label because it is missing.
 
How reproducible:

reproducible on systems using:
#> oc version
oc v3.7.0-alpha.1+b10446e-1184
kubernetes v1.7.6+a08f5eeb62
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://192.168.122.165:8443
openshift v3.7.0-alpha.1+b10446e-1184
kubernetes v1.7.6+a08f5eeb62

Steps to Reproduce:
1. Install OCP 3.7
2. Install Prometheus metrics
3. Query for metrics about a pod with specific uid

Actual results:
query:
container_cpu_system_seconds_total{container_name="POD",pod_name="registry-console-1-pf3dr"}
results:
0.01

query:
container_cpu_system_seconds_total{container_name="POD",pod_name="registry-console-1-pf3dr"}
results:
0.01

query:
container_cpu_system_seconds_total{container_name="POD",pod_uid="8a549ee4-b49a-11e7-b060-525400bd8690")
results:
NO RESULT

Expected results:

query:
container_cpu_system_seconds_total{container_name="POD",pod_name="registry-console-1-pf3dr"}
results:
0.01

query:
container_cpu_system_seconds_total{container_name="POD",pod_name="registry-console-1-pf3dr"}
results:
0.01

query:
container_cpu_system_seconds_total{container_name="POD",pod_uid="8a549ee4-b49a-11e7-b060-525400bd8690")
results:
0.01


Additional info:

when querying cAdvisor endpoint, the uid is already missing:

url:
https://prometheus.example.com:10250/metrics/cadvisor
response:
...
container_cpu_system_seconds_total{container_name="POD",id="/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c53e55a_b49a_11e7_b060_525400bd8690.slice/docker-99c275627fe90a5863bcf14eb6535b29b57438517c995243ddfe057fd0590714.scope",image="openshift/origin-pod:v3.7.0-alpha.1",name="k8s_POD_registry-console-1-pf3dr_default_8c53e55a-b49a-11e7-b060-525400bd8690_0",namespace="default",pod_name="registry-console-1-pf3dr"} 0.01
...