Bug 2027387 - [IBMCLOUD] Terraform ibmcloud-provider buffers entirely the qcow2 image causing spikes of 5GB of RAM during installation
Summary: [IBMCLOUD] Terraform ibmcloud-provider buffers entirely the qcow2 image causi...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.10
Hardware: x86_64
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.10.0
Assignee: aos-install
QA Contact: Pedro Amoedo
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-11-29 14:19 UTC by Pedro Amoedo
Modified: 2022-03-10 16:30 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-03-10 16:30:41 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 5436 0 None open Bug 2027387: fix(ibmcloud): Do not compute image file checksum 2021-12-01 16:43:13 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-10 16:30:57 UTC

Description Pedro Amoedo 2021-11-29 14:19:43 UTC 
Version:

$ openshift-install version
4.10 latest nightly, example "4.10.0-0.nightly-2021-11-28-164900"

Platform:

IBMCloud IPI

What happened?

openshift-install binary (specifically the terraform-ibmcloud-provider) for IBMCloud consumes an unusual amount of RAM compared with other cloudproviders:

- during initial state it reaches 2.5GB, which is already an unusual amount.
- during qcow2 image buffering it reaches it maximum spike of 5GB.
- after bootstrap it lowers down to the usual 200/300 MB like any other platform.

What did you expect to happen?

To maybe implement an image splitting strategy (like other providers) for the COS image bucket process to alleviate the memory consumption.

How to reproduce it (as minimally and precisely as possible)?

IBMCloud IPI installation with latest 4.10 nightly version

Anything else we need to know?

N/A
Comment 2 Pedro Amoedo 2021-11-30 19:39:46 UTC 
As a reference, "terraform-provider-vsphereprivate" calls "newLeaseUpdater"[1] function for the split image operation of the OVA file.

[1] - https://github.com/openshift/installer/blob/master/vendor/github.com/vmware/govmomi/nfc/lease_updater.go#L70

Best Regards.
Comment 3 Pedro Amoedo 2021-12-01 10:00:32 UTC 
FWIW, a different approach is the OpenStack "uploadBaseImage"[1] function thet creates a new image in Glance and uploads the RHCOS image there.

[1] - https://github.com/openshift/installer/blob/master/pkg/tfvars/openstack/rhcos_image.go#L21

Best Regards.
Comment 4 Pedro Amoedo 2021-12-01 18:46:38 UTC 
[pre-merge testing]

With the current PR[1], the installer (along with tf-ibm-provider) is not exceeding 300MB during all installations phases, initiation, COS bucket image creation, bootstrap, etc. So far so good.

[1] - https://github.com/openshift/installer/pull/5436

*** PASSED ***
Comment 9 errata-xmlrpc 2022-03-10 16:30:41 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056
Note You need to log in before you can comment on or make changes to this bug.