Bug 1829223

Summary: test: [sig-instrumentation] Prometheus when installed on the cluster should have a AlertmanagerReceiversNotConfigured alert in firing state [Suite:openshift/conformance/parallel]
Product: OpenShift Container Platform Reporter: Periklis Tsirakidis <periklis>
Component: MonitoringAssignee: Pawel Krupa <pkrupa>
Status: CLOSED ERRATA QA Contact: Junqi Zhao <juzhao>
Severity: high Docs Contact:
Priority: medium    
Version: 4.5CC: alegrand, anpicker, bparees, btofel, erooth, jesusr, kakkoyun, lcosic, mloibl, pkrupa, surbania
Target Milestone: ---Keywords: Reopened
Target Release: 4.5.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
test: [sig-instrumentation] Prometheus when installed on the cluster should have a AlertmanagerReceiversNotConfigured alert in firing state [Suite:openshift/conformance/parallel]
Last Closed: 2020-09-01 21:01:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Periklis Tsirakidis 2020-04-29 08:53:18 UTC
test: [sig-instrumentation] Prometheus when installed on the cluster should have a AlertmanagerReceiversNotConfigured alert in firing state [Suite:openshift/conformance/parallel] failed, see job: 

https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-aws-4.5/1598

Comment 3 Ben Parees 2020-04-30 01:35:18 UTC
This is also one of the top 10 failures across our CI system currently:
https://sippy-bparees.svc.ci.openshift.org/?release=4.5

raising severity accordingly.  I won't touch priority since that's determined by the team, but i'd like to see it higher too :)

Comment 4 Lili Cosic 2020-04-30 08:44:29 UTC
@Pawel if this is the case what I mentioned above, that it's the newly extended for that is causing this, I would suggest we just remove it from the tests must have alerts in firing state, and instead check that AlertmanagerReceiversNotConfigured is at least in pending state, wdyt?

Comment 8 Junqi Zhao 2020-05-06 09:37:23 UTC
tested with 4.5.0-0.nightly-2020-05-05-205255, the fix is fine

Comment 9 W. Trevor King 2020-05-07 23:38:08 UTC
*** Bug 1833134 has been marked as a duplicate of this bug. ***

Comment 10 errata-xmlrpc 2020-07-13 17:32:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409

Comment 11 Brett Tofel 2020-09-01 19:34:04 UTC
Reopening this bug as it seems like it still around in several 4.5 tests as of today. For instance:

https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_installer/4123/pull-ci-openshift-installer-release-4.5-e2e-aws/1300839262085713920

Comment 14 Ben Parees 2020-09-01 21:10:55 UTC
I also think Brett may have mixed up the tests.  In his link the failure is:
Prometheus when installed on the cluster shouldn't report any alerts in firing state apart from Watchdog and AlertmanagerReceiversNotConfigured


Which is not the same as the test this bug is/was for:
Prometheus when installed on the cluster should have a AlertmanagerReceiversNotConfigured alert in firing state


The test Brett pointed to is indeed failing quite a bit, but appears to currently be linked to https://bugzilla.redhat.com/show_bug.cgi?id=1847363