Bug 1893202 - e2e-operator flakes with "TestMetricsAccessible: prometheus returned unexpected results: timed out waiting for the condition"
Summary: e2e-operator flakes with "TestMetricsAccessible: prometheus returned unexpect...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-scheduler
Version: 4.5
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.5.z
Assignee: Mike Dame
QA Contact: RamaKasturi
URL:
Whiteboard:
Depends On: 1893201
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-10-30 14:34 UTC by Mike Dame
Modified: 2020-11-24 12:42 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1893201
Environment:
Last Closed: 2020-11-24 12:42:10 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-kube-scheduler-operator pull 299 0 None closed Bug 1893202: [release-4.5] Extend Prometheus E2E timeout 2021-01-13 21:16:49 UTC
Red Hat Product Errata RHSA-2020:5118 0 None None None 2020-11-24 12:42:56 UTC

Description Mike Dame 2020-10-30 14:34:22 UTC
+++ This bug was initially created as a clone of Bug #1893201 +++

The e2e-operator test frequently fails on a prometheus timeout:

 --- FAIL: TestMetricsAccessible (30.57s)
    scheduler_test.go:317: prometheus returned unexpected results: timed out waiting for the condition
FAIL
FAIL	github.com/openshift/cluster-kube-scheduler-operator/test/e2e	35.059s
FAIL

Example: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_cluster-kube-scheduler-operator/290/pull-ci-openshift-cluster-kube-scheduler-operator-release-4.5-e2e-aws-operator/1322155133357789184

We have tried extending the timeout for this test (https://github.com/openshift/cluster-kube-scheduler-operator/pull/299). It seems to fail for many runs before eventually passing

--- Additional comment from Mike Dame on 2020-10-30 14:33:45 UTC ---

Going to try backporting the change from #299 to release-4.5, where we're currently seeing the problem.

For QE, this bug should be verifiable by confirming that this test hasn't had any recent failures in CI

Comment 3 RamaKasturi 2020-11-17 14:44:50 UTC
Verified by looking at the link below and did not see any failure in last 24 hours, based on that moving bug to verified state.

https://search.ci.openshift.org/?search=scheduler_test.go%3A317%3A+prometheus+returned+unexpected+results%3A+timed+out+waiting+for+the+condition&maxAge=336h&context=1&type=bug%2Bjunit&name=&maxMatches=5&maxBytes=20971520&groupBy=job

Comment 5 errata-xmlrpc 2020-11-24 12:42:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.5.20 bug fix and golang security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5118


Note You need to log in before you can comment on or make changes to this bug.