Bug 1946597

Summary: Unnecessary series churn when a new version of kube-state-metrics is rolled out
Product: OpenShift Container Platform Reporter: Jan Fajerski <jfajersk>
Component: MonitoringAssignee: Jan Fajerski <jfajersk>
Status: CLOSED ERRATA QA Contact: Junqi Zhao <juzhao>
Severity: high Docs Contact:
Priority: high    
Version: 4.7CC: alegrand, anpicker, erooth, juzhao, kakkoyun, lcosic, pkrupa, spasquie, surbania
Target Milestone: ---   
Target Release: 4.6.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: 1945851 Environment:
Last Closed: 2021-04-27 14:20:49 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1945851    
Bug Blocks: 1931896    

Comment 2 Junqi Zhao 2021-04-16 01:49:11 UTC
tested with 4.6.0-0.nightly-2021-04-15-200627
pod label exists for kube_pod_info is expected, no instance label for it
search kube_pod_info{namespace="openshift-monitoring"}, result
      {
        "metric": {
          "__name__": "kube_pod_info",
          "container": "kube-rbac-proxy-main",
          "created_by_kind": "StatefulSet",
          "created_by_name": "prometheus-k8s",
          "endpoint": "https-main",
          "host_ip": "10.0.32.3",
          "job": "kube-state-metrics",
          "namespace": "openshift-monitoring",
          "node": "qeci-19120-hsqfv-worker-a-rd8dz.c.openshift-qe.internal",
          "pod": "prometheus-k8s-0",
          "pod_ip": "10.131.0.35",
          "priority_class": "openshift-user-critical",
          "service": "kube-state-metrics",
          "uid": "169f1be3-de29-4f40-bc67-843b10344140"
        },


kube_pod_info{container="kube-rbac-proxy-main", created_by_kind="DaemonSet", created_by_name="node-exporter", endpoint="https-main", host_ip="10.0.130.185", job="kube-state-metrics", namespace="openshift-monitoring", node="ip-10-0-130-185.us-east-2.compute.internal", pod="node-exporter-x2bw5", pod_ip="10.0.130.185", priority_class="system-cluster-critical", service="kube-state-metrics", uid="c407cd8e-de07-4c10-b203-935874306853"}

the instance and pod labels are absent for other kube_* metrics, example
kube_node_role
      {
        "metric": {
          "__name__": "kube_node_role",
          "container": "kube-rbac-proxy-main",
          "endpoint": "https-main",
          "job": "kube-state-metrics",
          "namespace": "openshift-monitoring",
          "node": "qeci-19120-hsqfv-worker-c-64tjw.c.openshift-qe.internal",
          "role": "worker",
          "service": "kube-state-metrics"
        },

Comment 5 errata-xmlrpc 2021-04-27 14:20:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.26 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1232