Bug 2027387

Summary:	[IBMCLOUD] Terraform ibmcloud-provider buffers entirely the qcow2 image causing spikes of 5GB of RAM during installation
Product:	OpenShift Container Platform	Reporter:	Pedro Amoedo <pamoedom>
Component:	Installer	Assignee:	aos-install
Installer sub component:	openshift-installer	QA Contact:	Pedro Amoedo <pamoedom>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	medium
Priority:	unspecified	CC:	cschaefe, jnowicki
Version:	4.10
Target Milestone:	---
Target Release:	4.10.0
Hardware:	x86_64
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-03-10 16:30:41 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Pedro Amoedo 2021-11-29 14:19:43 UTC

Version:

$ openshift-install version
4.10 latest nightly, example "4.10.0-0.nightly-2021-11-28-164900"

Platform:

IBMCloud IPI

What happened?

openshift-install binary (specifically the terraform-ibmcloud-provider) for IBMCloud consumes an unusual amount of RAM compared with other cloudproviders:

- during initial state it reaches 2.5GB, which is already an unusual amount.
- during qcow2 image buffering it reaches it maximum spike of 5GB.
- after bootstrap it lowers down to the usual 200/300 MB like any other platform.

What did you expect to happen?

To maybe implement an image splitting strategy (like other providers) for the COS image bucket process to alleviate the memory consumption.

How to reproduce it (as minimally and precisely as possible)?

IBMCloud IPI installation with latest 4.10 nightly version

Anything else we need to know?

N/A

Comment 2 Pedro Amoedo 2021-11-30 19:39:46 UTC

As a reference, "terraform-provider-vsphereprivate" calls "newLeaseUpdater"[1] function for the split image operation of the OVA file.

[1] - https://github.com/openshift/installer/blob/master/vendor/github.com/vmware/govmomi/nfc/lease_updater.go#L70

Best Regards.

Comment 3 Pedro Amoedo 2021-12-01 10:00:32 UTC

FWIW, a different approach is the OpenStack "uploadBaseImage"[1] function thet creates a new image in Glance and uploads the RHCOS image there.

[1] - https://github.com/openshift/installer/blob/master/pkg/tfvars/openstack/rhcos_image.go#L21

Best Regards.

Comment 4 Pedro Amoedo 2021-12-01 18:46:38 UTC

[pre-merge testing]

With the current PR[1], the installer (along with tf-ibm-provider) is not exceeding 300MB during all installations phases, initiation, COS bucket image creation, bootstrap, etc. So far so good.

[1] - https://github.com/openshift/installer/pull/5436

*** PASSED ***

Comment 9 errata-xmlrpc 2022-03-10 16:30:41 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056