Bug 1717619 - Telemetry should include the condition reason on degraded operators
Summary: Telemetry should include the condition reason on degraded operators
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Cluster Version Operator
Version: 4.1.z
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.1.z
Assignee: Clayton Coleman
QA Contact: liujia
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-06-05 19:56 UTC by Clayton Coleman
Modified: 2019-07-04 09:01 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-07-04 09:01:24 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:1635 0 None None None 2019-07-04 09:01:33 UTC

Description Clayton Coleman 2019-06-05 19:56:08 UTC
This bug was initially created as a copy of Bug #1717617

I am copying this bug because: it should be part of 4.1.z so that we can triage failures.

While assessing 4.1.0 GA degraded operators Adam suggested capturing the reason (which has bounded cardinality) on the telemetry metric for cluster_operator_conditions, which would

1. incentivize teams to add good reasons
2. allow a quick summarization of why the operator is degraded

Added in https://github.com/openshift/cluster-version-operator/pull/197, should be back ported to 4.1.1

Comment 2 liujia 2019-06-20 08:43:53 UTC
Version: 4.1.0-0.nightly-2019-06-19-220253

According to pr197, Checked from prometheus console that "reason" section was added into cluster_operator_conditions metric now.

cluster_operator_conditions{condition="Available",endpoint="metrics",instance="10.0.131.249:9099",job="cluster-version-operator",name="image-registry",namespace="openshift-cluster-version",pod="cluster-version-operator-94456444f-clw9n",reason="Ready",service="cluster-version-operator"}
...
cluster_operator_conditions{condition="Degraded",endpoint="metrics",instance="10.0.131.249:9099",job="cluster-version-operator",name="cloud-credential",namespace="openshift-cluster-version",pod="cluster-version-operator-94456444f-clw9n",reason="NoCredentialsFailing",service="cluster-version-operator"}
...
cluster_operator_conditions{condition="Progressing",endpoint="metrics",instance="10.0.131.249:9099",job="cluster-version-operator",name="cloud-credential",namespace="openshift-cluster-version",pod="cluster-version-operator-94456444f-clw9n",reason="ReconcilingComplete",service="cluster-version-operator"}
...
cluster_operator_conditions{condition="Upgradeable",endpoint="metrics",instance="10.0.131.249:9099",job="cluster-version-operator",name="kube-controller-manager",namespace="openshift-cluster-version",pod="cluster-version-operator-94456444f-clw9n",reason="AsExpected",service="cluster-version-operator"}

Verify the bug according to my understanding, please feel free to re-open it if it's not expected.

Comment 4 errata-xmlrpc 2019-07-04 09:01:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1635


Note You need to log in before you can comment on or make changes to this bug.