Bug 1823261 - [4.4]No prometheus graph for firing alerts except Watchdog
Summary: [4.4]No prometheus graph for firing alerts except Watchdog
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Management Console
Version: 4.4
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.4.0
Assignee: David Taylor
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-13 04:31 UTC by Junqi Zhao
Modified: 2020-05-04 11:49 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of: 1771843
Environment:
Last Closed: 2020-05-04 11:49:15 UTC
Target Upstream Version:


Attachments (Terms of Use)
there is diagram for Watchdog alert (100.60 KB, image/png)
2020-04-13 04:35 UTC, Junqi Zhao
no flags Details
AlertmanagerReceiversNotConfigured with Prometheus graph (136.81 KB, image/png)
2020-04-13 18:21 UTC, David Taylor
no flags Details
alert other than Watchdog is showing prometheus graph (4.4) (132.40 KB, image/png)
2020-04-13 18:34 UTC, David Taylor
no flags Details
could show metrics diagram for AlertmanagerReceiversNotConfigured alert (89.46 KB, image/png)
2020-04-14 04:04 UTC, Junqi Zhao
no flags Details
No prometheus graph for AlertmanagerReceiversNotConfigured on UPI baremetal cluster (110.75 KB, image/png)
2020-04-15 10:41 UTC, Junqi Zhao
no flags Details
API returns value successfully, but not shown on UI (272.01 KB, image/png)
2020-04-15 10:48 UTC, Junqi Zhao
no flags Details
chart shown in 4.4-rc.8 (141.34 KB, image/png)
2020-04-15 14:56 UTC, David Taylor
no flags Details
upi-on-vsphere cluster, alerts other than Watchdog could show chart (114.05 KB, image/png)
2020-04-16 03:20 UTC, Junqi Zhao
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:0581 None None None 2020-05-04 11:49:51 UTC

Comment 1 Junqi Zhao 2020-04-13 04:32:51 UTC
UPI on vSphere cluster, payload: 4.4.0-0.nightly-2020-04-09-220855

Comment 2 Junqi Zhao 2020-04-13 04:35:48 UTC
Created attachment 1678367 [details]
there is diagram for Watchdog alert

Comment 3 David Taylor 2020-04-13 18:21:22 UTC
Created attachment 1678499 [details]
AlertmanagerReceiversNotConfigured with Prometheus graph

Comment 4 David Taylor 2020-04-13 18:34:44 UTC
Created attachment 1678501 [details]
alert other than Watchdog is showing prometheus graph (4.4)

Comment 5 David Taylor 2020-04-13 18:46:17 UTC
Hi 
Using 4.4.0-0.ci-2020-04-13-142213 I do see a Prometheus graph for AlertmanagerReceiversNotConfigured (attachment 1678501 [details]).  I also see a Prometheus graph in 4.5.0-0.ci-2020-04-13-003218 (attachment 1678499 [details]).

Not sure if just not working in 4.4.0-0.nightly-2020-04-09-220855, or something to do with 'UPI on vSphere cluster'?

Comment 6 Junqi Zhao 2020-04-14 04:03:51 UTC
Tested with 4.4.0-0.nightly-2020-04-13-113747, it is IPI AWS cluster, there is not such issue now, see the attached picture, maybe the error is just in few payload

Comment 7 Junqi Zhao 2020-04-14 04:04:34 UTC
Created attachment 1678614 [details]
could show metrics diagram for AlertmanagerReceiversNotConfigured alert

Comment 10 Junqi Zhao 2020-04-15 10:41:16 UTC
Created attachment 1679001 [details]
No prometheus graph for AlertmanagerReceiversNotConfigured on UPI baremetal cluster

Comment 11 Junqi Zhao 2020-04-15 10:48:09 UTC
checked, the API returns value successfully, but not shown on UI, see the picture

Comment 12 Junqi Zhao 2020-04-15 10:48:33 UTC
Created attachment 1679003 [details]
API returns value successfully, but not shown on UI

Comment 13 bpeterse 2020-04-15 13:35:31 UTC
This appears to have been fixed in 4.3 and should be working now.

Comment 14 bpeterse 2020-04-15 13:44:06 UTC
Misread the requirement for vsphere.  Back to assigned.

Comment 15 David Taylor 2020-04-15 14:56:35 UTC
Created attachment 1679098 [details]
chart shown in 4.4-rc.8

Comment 16 David Taylor 2020-04-15 14:59:33 UTC
Hi, issued '@cluster-bot launch 4.4 vsphere' which resulted in a 'stable 4.4.0-rc.8' cluster.
I do see a prometheus graph for AlertmanagerReceiversNotConfigured, see attachment 1679098 [details]

Comment 17 Junqi Zhao 2020-04-16 02:10:53 UTC
https://openshift-release.svc.ci.openshift.org/releasestream/4-stable/release/4.4.0-rc.8
Release 4.4.0-rc.8 was created from registry.svc.ci.openshift.org/ocp/release:4.4.0-0.nightly-2020-04-13-113747

see Comment 9, our payload is 4.4.0-0.nightly-2020-04-14-222426, it is newer than 4.4.0-rc.8.

Will try to install a new payload, if it still happen, will assign back

Comment 18 Junqi Zhao 2020-04-16 03:18:11 UTC
upi-on-vsphere 4.4.0-0.nightly-2020-04-15-212458 cluster, no such issue there, maybe the issue is only with a few payload, close it first

Comment 19 Junqi Zhao 2020-04-16 03:20:00 UTC
Created attachment 1679208 [details]
upi-on-vsphere cluster, alerts other than Watchdog could show chart

Comment 22 errata-xmlrpc 2020-05-04 11:49:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581


Note You need to log in before you can comment on or make changes to this bug.