Bug 1684431 - KubeVersionMismatch: There are 2 different versions of Kubernetes components running in Alerts
Summary: KubeVersionMismatch: There are 2 different versions of Kubernetes components ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Monitoring
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.1.0
Assignee: Frederic Branczyk
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-01 09:18 UTC by Neeraj
Modified: 2019-06-04 10:44 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:44:51 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 None None None 2019-06-04 10:44:58 UTC

Description Neeraj 2019-03-01 09:18:49 UTC
Description of problem:
There is an Alert in the OpenShift console that reads "KubeVersionMismatch: There are 2 different versions of Kubernetes components running.". This was just after installation and without any customisation from my side.

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:

```
$ openshift-install version
openshift-install v0.12.0
$ openshift-install --dir=taakrsi4-demo3/ create cluster
```

Prometheus (query "kubernetes_build_info{job!="kube-dns"}") reports:

```
kubernetes_build_info{buildDate="2019-01-29T23:36:59Z",compiler="gc",endpoint="https",gitCommit="f39ab668d3",gitTreeState="clean",gitVersion="v1.12.4+f39ab668d3",goVersion="go1.10.3",instance="10.0.27.179:10251",job="scheduler",major="1",minor="12+",namespace="openshift-kube-scheduler",platform="linux/amd64",pod="openshift-kube-scheduler-ip-10-0-27-179.eu-central-1.compute.internal",service="scheduler"}	1
kubernetes_build_info{buildDate="2019-01-29T23:36:59Z",compiler="gc",endpoint="https",gitCommit="f39ab668d3",gitTreeState="clean",gitVersion="v1.12.4+f39ab668d3",goVersion="go1.10.3",instance="10.0.27.179:10252",job="kube-controller-manager",major="1",minor="12+",namespace="openshift-kube-controller-manager",platform="linux/amd64",pod="openshift-kube-controller-manager-ip-10-0-27-179.eu-central-1.compute.internal",service="kube-controller-manager"}	1
kubernetes_build_info{buildDate="2019-01-29T23:36:59Z",compiler="gc",endpoint="https",gitCommit="f39ab668d3",gitTreeState="clean",gitVersion="v1.12.4+f39ab668d3",goVersion="go1.10.3",instance="10.0.47.122:10251",job="scheduler",major="1",minor="12+",namespace="openshift-kube-scheduler",platform="linux/amd64",pod="openshift-kube-scheduler-ip-10-0-47-122.eu-central-1.compute.internal",service="scheduler"}	1
kubernetes_build_info{buildDate="2019-01-29T23:36:59Z",compiler="gc",endpoint="https",gitCommit="f39ab668d3",gitTreeState="clean",gitVersion="v1.12.4+f39ab668d3",goVersion="go1.10.3",instance="10.0.47.122:10252",job="kube-controller-manager",major="1",minor="12+",namespace="openshift-kube-controller-manager",platform="linux/amd64",pod="openshift-kube-controller-manager-ip-10-0-47-122.eu-central-1.compute.internal",service="kube-controller-manager"}	1
kubernetes_build_info{buildDate="2019-01-29T23:36:59Z",compiler="gc",endpoint="https",gitCommit="f39ab668d3",gitTreeState="clean",gitVersion="v1.12.4+f39ab668d3",goVersion="go1.10.3",instance="10.0.9.56:10251",job="scheduler",major="1",minor="12+",namespace="openshift-kube-scheduler",platform="linux/amd64",pod="openshift-kube-scheduler-ip-10-0-9-56.eu-central-1.compute.internal",service="scheduler"}	1
kubernetes_build_info{buildDate="2019-01-29T23:36:59Z",compiler="gc",endpoint="https",gitCommit="f39ab668d3",gitTreeState="clean",gitVersion="v1.12.4+f39ab668d3",goVersion="go1.10.3",instance="10.0.9.56:10252",job="kube-controller-manager",major="1",minor="12+",namespace="openshift-kube-controller-manager",platform="linux/amd64",pod="openshift-kube-controller-manager-ip-10-0-9-56.eu-central-1.compute.internal",service="kube-controller-manager"}	1
kubernetes_build_info{buildDate="2019-02-01T23:06:11Z",compiler="gc",endpoint="https-metrics",gitCommit="91d94b671b",gitTreeState="clean",gitVersion="v1.12.4+91d94b671b",goVersion="go1.10.3",instance="10.0.135.74:10250",job="kubelet",major="1",minor="12+",namespace="kube-system",node="ip-10-0-135-74.eu-central-1.compute.internal",platform="linux/amd64",service="kubelet"}	1
kubernetes_build_info{buildDate="2019-02-01T23:06:11Z",compiler="gc",endpoint="https-metrics",gitCommit="91d94b671b",gitTreeState="clean",gitVersion="v1.12.4+91d94b671b",goVersion="go1.10.3",instance="10.0.145.112:10250",job="kubelet",major="1",minor="12+",namespace="kube-system",node="ip-10-0-145-112.eu-central-1.compute.internal",platform="linux/amd64",service="kubelet"}	1
kubernetes_build_info{buildDate="2019-02-01T23:06:11Z",compiler="gc",endpoint="https-metrics",gitCommit="91d94b671b",gitTreeState="clean",gitVersion="v1.12.4+91d94b671b",goVersion="go1.10.3",instance="10.0.161.79:10250",job="kubelet",major="1",minor="12+",namespace="kube-system",node="ip-10-0-161-79.eu-central-1.compute.internal",platform="linux/amd64",service="kubelet"}	1
kubernetes_build_info{buildDate="2019-02-01T23:06:11Z",compiler="gc",endpoint="https-metrics",gitCommit="91d94b671b",gitTreeState="clean",gitVersion="v1.12.4+91d94b671b",goVersion="go1.10.3",instance="10.0.27.179:10250",job="kubelet",major="1",minor="12+",namespace="kube-system",node="ip-10-0-27-179.eu-central-1.compute.internal",platform="linux/amd64",service="kubelet"}	1
kubernetes_build_info{buildDate="2019-02-01T23:06:11Z",compiler="gc",endpoint="https-metrics",gitCommit="91d94b671b",gitTreeState="clean",gitVersion="v1.12.4+91d94b671b",goVersion="go1.10.3",instance="10.0.47.122:10250",job="kubelet",major="1",minor="12+",namespace="kube-system",node="ip-10-0-47-122.eu-central-1.compute.internal",platform="linux/amd64",service="kubelet"}	1
kubernetes_build_info{buildDate="2019-02-01T23:06:11Z",compiler="gc",endpoint="https-metrics",gitCommit="91d94b671b",gitTreeState="clean",gitVersion="v1.12.4+91d94b671b",goVersion="go1.10.3",instance="10.0.9.56:10250",job="kubelet",major="1",minor="12+",namespace="kube-system",node="ip-10-0-9-56.eu-central-1.compute.internal",platform="linux/amd64",service="kubelet"}
```
Actual results:

There are lot of waring of kubeversion mismatch due to different git Versions

Expected results:

A newly installed cluster does not show any open alerts out-of-the-box

Comment 1 Junqi Zhao 2019-03-04 03:10:43 UTC
Yes, it had this issue before, but should not be a monitoring issue, we just throw out the alerts, anyway, there is not such issue with
4.0.0-0.nightly-2019-02-28-054829

The payload you use maybe too old
check via
# oc get clusterversion

Comment 3 lserven 2019-03-27 14:50:04 UTC
We should be resolved as of February 7th thanks to https://github.com/openshift/cluster-monitoring-operator/pull/235. If we are still seeing this issue today, then it is a bug in the alerting rule. Junqi, can you please verify if the alert is still showing today?

-Lucas

Comment 6 errata-xmlrpc 2019-06-04 10:44:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.