Bug 1893202

Summary:	e2e-operator flakes with "TestMetricsAccessible: prometheus returned unexpected results: timed out waiting for the condition"
Product:	OpenShift Container Platform	Reporter:	Mike Dame <mdame>
Component:	kube-scheduler	Assignee:	Mike Dame <mdame>
Status:	CLOSED ERRATA	QA Contact:	RamaKasturi <knarra>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	4.5	CC:	aos-bugs, knarra, maszulik, mfojtik
Target Milestone:	---
Target Release:	4.5.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1893201	Environment:
Last Closed:	2020-11-24 12:42:10 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1893201
Bug Blocks:

Description Mike Dame 2020-10-30 14:34:22 UTC

+++ This bug was initially created as a clone of Bug #1893201 +++

The e2e-operator test frequently fails on a prometheus timeout:

 --- FAIL: TestMetricsAccessible (30.57s)
    scheduler_test.go:317: prometheus returned unexpected results: timed out waiting for the condition
FAIL
FAIL	github.com/openshift/cluster-kube-scheduler-operator/test/e2e	35.059s
FAIL

Example: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_cluster-kube-scheduler-operator/290/pull-ci-openshift-cluster-kube-scheduler-operator-release-4.5-e2e-aws-operator/1322155133357789184

We have tried extending the timeout for this test (https://github.com/openshift/cluster-kube-scheduler-operator/pull/299). It seems to fail for many runs before eventually passing

--- Additional comment from Mike Dame on 2020-10-30 14:33:45 UTC ---

Going to try backporting the change from #299 to release-4.5, where we're currently seeing the problem.

For QE, this bug should be verifiable by confirming that this test hasn't had any recent failures in CI

Comment 3 RamaKasturi 2020-11-17 14:44:50 UTC

Verified by looking at the link below and did not see any failure in last 24 hours, based on that moving bug to verified state.

https://search.ci.openshift.org/?search=scheduler_test.go%3A317%3A+prometheus+returned+unexpected+results%3A+timed+out+waiting+for+the+condition&maxAge=336h&context=1&type=bug%2Bjunit&name=&maxMatches=5&maxBytes=20971520&groupBy=job

Comment 5 errata-xmlrpc 2020-11-24 12:42:10 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.5.20 bug fix and golang security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5118