Bug 1958959 - unneeded CCO alert already covered by CVO
Summary: unneeded CCO alert already covered by CVO
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Cloud Credential Operator
Version: 4.7
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.6.z
Assignee: Joel Diaz
QA Contact: wang lin
URL:
Whiteboard:
Depends On: 1948702
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-05-10 13:43 UTC by OpenShift BugZilla Robot
Modified: 2021-06-08 13:54 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Removed functionality
Doc Text:
The cluster-version operator is responsible for reporting if the cloud-credential-operator's deployment is unhealthy; no need for the cloud-credential-operator to handle this directly (resulting in doubling reporting if there's an issue).
Clone Of:
Environment:
Last Closed: 2021-06-08 13:54:19 UTC
Target Upstream Version:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cloud-credential-operator pull 337 0 None open [release-4.6] Bug 1958959: [release-4.7] manifests/0000_90_cloud-credential-operator_04_alertrules: Drop CloudCredential... 2021-05-10 14:03:35 UTC
Red Hat Product Errata RHBA-2021:2157 0 None None None 2021-06-08 13:54:39 UTC

Comment 2 wang lin 2021-05-27 09:17:48 UTC 
Verified this bug using the built by cluster-bot with PR https://github.com/openshift/cloud-credential-operator/pull/337 merged.

$ oc get clusterversion
NAME      VERSION                                                  AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.0-0.ci.test-2021-05-27-080846-ci-ln-h79wt0k-latest   True        False         4m56s   Cluster version is 4.6.0-0.ci.test-2021-05-27-080846-ci-ln-h79wt0k-latest

1. Login to prometheus console, check CloudCredentialOperatorDown has remove from CloudCredentialOperator

2. Create an invalid cr request, check cco is degraded
$ oc get co cloud-credential
NAME               VERSION                                                  AVAILABLE   PROGRESSING   DEGRADED   SINCE
cloud-credential   4.6.0-0.ci.test-2021-05-27-080846-ci-ln-h79wt0k-latest   True        True          True       41m

3.check when cco down, cvo will fire the alerts
    {
      "labels": {
        "alertname": "ClusterOperatorDegraded",
        "condition": "Degraded",
        "endpoint": "metrics",
        "instance": "10.0.187.47:9099",
        "job": "cluster-version-operator",
        "name": "cloud-credential",
        "namespace": "openshift-cluster-version",
        "pod": "cluster-version-operator-5595c5fdd6-tbvst",
        "prometheus": "openshift-monitoring/k8s",
        "reason": "CredentialsFailing",
        "service": "cluster-version-operator",
        "severity": "critical"
      },
      "annotations": {
        "message": "Cluster operator cloud-credential has been degraded for 10 minutes. Operator is degraded because CredentialsFailing and cluster upgrades will be unstable."
      },
      "startsAt": "2021-05-27T09:09:59.213Z",
      "endsAt": "2021-05-27T09:18:29.213Z",
      "generatorURL": "https://prometheus-k8s-openshift-monitoring.apps.ci-ln-h79wt0k-d5d6b.origin-ci-int-aws.dev.rhcloud.com/graph?g0.expr=cluster_operator_conditions%7Bcondition%3D%22Degraded%22%2Cjob%3D%22cluster-version-operator%22%7D+%3D%3D+1&g0.tab=1",
      "status": {
        "state": "active",
        "silencedBy": [],
        "inhibitedBy": []
      },
      "receivers": [
        "Critical"
      ],
      "fingerprint": "7e911a8af46a627e"
    },
    {
      "labels": {
        "alertname": "ClusterOperatorDown",
        "endpoint": "metrics",
        "instance": "10.0.187.47:9099",
        "job": "cluster-version-operator",
        "name": "cloud-credential",
        "namespace": "openshift-cluster-version",
        "pod": "cluster-version-operator-5595c5fdd6-tbvst",
        "prometheus": "openshift-monitoring/k8s",
        "service": "cluster-version-operator",
        "severity": "critical",
        "version": "4.6.0-0.ci.test-2021-05-27-080846-ci-ln-h79wt0k-latest"
      },
      "annotations": {
        "message": "Cluster operator cloud-credential has not been available for 10 minutes. Operator may be down or disabled, cluster will not be kept up to date and upgrades will not be possible."
      },
      "startsAt": "2021-05-27T09:09:59.213Z",
      "endsAt": "2021-05-27T09:18:29.213Z",
      "generatorURL": "https://prometheus-k8s-openshift-monitoring.apps.ci-ln-h79wt0k-d5d6b.origin-ci-int-aws.dev.rhcloud.com/graph?g0.expr=cluster_operator_up%7Bjob%3D%22cluster-version-operator%22%7D+%3D%3D+0&g0.tab=1",
      "status": {
        "state": "active",
        "silencedBy": [],
        "inhibitedBy": []
      },
      "receivers": [
        "Critical"
      ],
      "fingerprint": "9e847827c26ec62a"
    },


##
the bug is pre-merge verified. After the PR gets merged, the bug will be moved to VERIFIED by the bot automatically or, if not working, by me manually.
Comment 8 errata-xmlrpc 2021-06-08 13:54:19 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.32 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2157
Note You need to log in before you can comment on or make changes to this bug.