Bug 1815944 - e2e tests use substantially more CPU in CI infrastructure than necessary
Summary: e2e tests use substantially more CPU in CI infrastructure than necessary
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Test Infrastructure
Version: 4.5
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.3.z
Assignee: Clayton Coleman
QA Contact:
URL:
Whiteboard:
Depends On: 1815941 1815943
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-03-22 22:33 UTC by Clayton Coleman
Modified: 2023-09-14 05:54 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1815943
Environment:
Last Closed: 2020-04-08 07:39:52 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:1262 0 None None None 2020-04-08 07:39:56 UTC

Description Clayton Coleman 2020-03-22 22:33:08 UTC
+++ This bug was initially created as a clone of Bug #1815943 +++

+++ This bug was initially created as a clone of Bug #1815941 +++

A performance issue was identified in Ginkgo where every Openshift-tests invocation was triggering a full stack trace for every test, every time a test ran.  We added a carry patch to our forked ginkgo (which is mostly static) to disable capturing this trace and reduced total CPU used by an e2e run for 4.5 from ~10.5k core seconds to ~4.5k core seconds (60% reduction in total CPU). As e2e is one of our more significant loads this has a significant reduction in CI resources consumed.

The fix is safe to use within our limited use of Ginkgo and has been backported to 4.5 and 4.4 already with a vendor bump.  We should vendor bump in 4.3 as well since that's a significant amount of installs.

--- Additional comment from Clayton Coleman on 2020-03-22 18:31:48 EDT ---

Ginkgo patch in https://github.com/openshift/onsi-ginkgo/commit/67da0dd32db383f566d9d2b6559bf1d0ef03b5cb

Vendor bump in 4.5: https://github.com/openshift/origin/pull/24742 
Vendor bump in 4.4: https://github.com/openshift/origin/pull/24743
Vendor bump in 4.3: https://github.com/openshift/origin/pull/24746

Verified in 4.5 and 4.4 builds, graph of pod core seconds from api.ci (the tall job at the same time is a 4.3 build)

https://www.dropbox.com/s/jrpw3bsxclqcb64/Screenshot%202020-03-22%2018.30.38.png?raw=1

Comment 6 errata-xmlrpc 2020-04-08 07:39:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:1262

Comment 7 Red Hat Bugzilla 2023-09-14 05:54:35 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days


Note You need to log in before you can comment on or make changes to this bug.