Bug 1926795

Summary: CannotRetrieveUpdates alerts on Critical severity
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: Cluster Version OperatorAssignee: Jack Ottofaro <jack.ottofaro>
Status: CLOSED ERRATA QA Contact: Yang Yang <yanyang>
Severity: high Docs Contact:
Priority: medium    
Version: 4.6.zCC: aos-bugs, jokerman, lmohanty, wking, yanyang
Target Milestone: ---   
Target Release: 4.7.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-03-25 01:53:01 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1926310    
Bug Blocks:    

Comment 1 W. Trevor King 2021-02-09 16:18:46 UTC 
Not a blocker, because the alert landed as critical in 4.6 [1], and we don't want to delay 4.7's GA on something that is not a 4.6->4.7 regression.

[1]: https://github.com/openshift/cluster-version-operator/blob/94358e80fa8151e43690d5925abc9337e90dca1c/install/0000_90_cluster-version-operator_02_servicemonitor.yaml#L47-L53
Comment 3 Yang Yang 2021-03-11 05:53:55 UTC 
Verified with 4.7.0-0.nightly-2021-03-10-202622

Steps to verify:

1. Install a cluster with 4.7.0-0.nightly-2021-03-10-202622
2. After 1 hour, check CannotRetrieveUpdates alert

[root@preserve-yangyangmerrn-1 tmp]# token=`oc -n openshift-monitoring sa get-token prometheus-k8s` 
[root@preserve-yangyangmerrn-1 tmp]# curl -s -k -H "Authorization: Bearer $token" https://prometheus-k8s-openshift-monitoring.apps.yangyang0311.qe.gcp.devcluster.openshift.com/api/v1/alerts | jq '.data.alerts[]|select(.labels.alertname == "CannotRetrieveUpdates")'
{
  "labels": {
    "alertname": "CannotRetrieveUpdates",
    "endpoint": "metrics",
    "instance": "10.0.0.3:9099",
    "job": "cluster-version-operator",
    "namespace": "openshift-cluster-version",
    "pod": "cluster-version-operator-58cfdd7895-zgdrm",
    "service": "cluster-version-operator",
    "severity": "warning"        <------ severity is changed to warning
  },
  "annotations": {
    "message": "Cluster version operator has not retrieved updates in 1h 6m 4s. Failure reason VersionNotFound .  For more information refer to https://console-openshift-console.apps.yangyang0311.qe.gcp.devcluster.openshift.com/settings/cluster/."
  },
  "state": "firing",
  "activeAt": "2021-03-11T05:45:16.457044785Z",
  "value": "3.964457000017166e+03"
}

Moving it to verified state.
Comment 6 errata-xmlrpc 2021-03-25 01:53:01 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.7.3 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:0821