Bug 1925343 - [ci] e2e-metal tests are not using reserved instances
Summary: [ci] e2e-metal tests are not using reserved instances
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.7
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.8.0
Assignee: Matthew Staebler
QA Contact: Gaoyun Pei
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-02-04 22:24 UTC by Matthew Staebler
Modified: 2021-07-27 22:42 UTC (History)
0 users

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-27 22:41:34 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 4938 0 None open bug 1925343: upi/metal: add more configuration options for bare metal 2021-05-18 00:50:19 UTC
Github openshift release pull 15880 0 None open bug 1925343: e2e-metal: use reserved instances for tests 2021-02-17 20:20:20 UTC
Red Hat Product Errata RHSA-2021:2438 0 None None None 2021-07-27 22:42:14 UTC

Description Matthew Staebler 2021-02-04 22:24:03 UTC
The e2e-metal tests should be using the instances in the ny5 datacenter that we have reserved. Currently, the tests are running on any datacenter and on any available hardware.

Comment 1 Scott Dodson 2021-02-05 13:19:57 UTC
Here's an example commit from when I tested reserved instances in the past, I know I mentioned this on slack but in case someone other than Matthew ends up working on this bug here it is.

https://github.com/openshift/installer/commit/66c3fee9731116ed70c1a9fcad3ad6eaa4dc905b

I believe in order to do what we're asked we need to change the facilities to ['ny5'], the plan to 'c3.small.x86', and hardware_reservation_id = "next-available"

There's only 39 (weird number?) reserved instances, so with 7 hosts per cluster (1 bootstrap, 3 control-plane, 3 worker) that only leaves enough for five clusters. We don't currently tear down the bootstrap node once it's done, maybe we should fix that. If we were that would probably enable us to run six concurrent clusters in this reserved pool rather than five. We may also want to look at boskos leases just to ensure that we don't attempt to start more than we want at once.

The terraform code in the installer repo was meant to be generic enough that it could be used outside of our specific CI jobs and that's why most of these are variables so it's worth considering whether that's worth splitting the config between installer and release repo.

Comment 2 Scott Dodson 2021-02-05 13:20:48 UTC
Oh, I also don't believe this is urgent so feel free to lower that to high/high if you wish.

Comment 4 Matthew Staebler 2021-04-19 17:23:06 UTC
We'll look to progress further on this next sprint.

Comment 7 Gaoyun Pei 2021-06-13 06:50:36 UTC
This is a CI issue for adding packet_facility, packet_plan and packet_hardware_reservation_id variables to the terraform templates, so we could use the reserved instances in e2e-metal CI testing.

Checked the e2e-metal-4.8 test in latest two payloads: 4.8.0-0.nightly-2021-06-12-174011, 4.8.0-0.nightly-2021-06-12-223426, both succeeded as below, move this bug as VERIFIED.

https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-metal-4.8/1403844375636611072
https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-metal-4.8/1403769632917032960

Comment 10 errata-xmlrpc 2021-07-27 22:41:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438


Note You need to log in before you can comment on or make changes to this bug.