Bug 2006360 - Origin should run k8s and OpenShift tests separately
Summary: Origin should run k8s and OpenShift tests separately
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Test Framework
Version: 4.9
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.10.0
Assignee: Oleg Bulatov
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 2006362
TreeView+ depends on / blocked
 
Reported: 2021-09-21 14:24 UTC by Stephen Benjamin
Modified: 2021-09-27 13:16 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2006362 (view as bug list)
Environment:
Last Closed: 2021-09-27 13:16:20 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Stephen Benjamin 2021-09-21 14:24:47 UTC
During a recent TRT stand-up, David pointed out the extremely high pass rates
of upstream kube tests, but those same tests in OpenShift fail quite a
bit. There's a theory that perhaps some of the OpenShift tests like
builds use up so much I/O and other resources that they introduce
instability.

In master we have a change that runs them separately, and it seems to have improved the situation overall.  4.10 today has a pass rate of 70% of all jobs running e2e tests, but 4.9 is around 51%.  This is visible in the 4th release indicator widget on the overview page:  https://sippy.ci.openshift.org/sippy-ng/release/4.10 vs https://sippy.ci.openshift.org/sippy-ng/release/4.9

It's not clear if all or any of that is attributable to that change, but we'd like to backport it to 4.9 to confirm.


Note You need to log in before you can comment on or make changes to this bug.