Bug 2077975

Summary:	logging large range requests in < etcd-3.4.16 consumes significant memory
Product:	OpenShift Container Platform	Reporter:	Scott Dodson <sdodson>
Component:	Etcd	Assignee:	melbeher
Status:	CLOSED DEFERRED	QA Contact:	ge liu <geliu>
Severity:	high	Docs Contact:
Priority:	medium
Version:	4.6	CC:	melbeher, tjungblu
Target Milestone:	---
Target Release:	4.8.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-09-08 13:31:50 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2077501
Bug Blocks:	2064371

Description Scott Dodson 2022-04-22 18:49:38 UTC

Description of problem:
The logging function which produces the large range log messages unmarshals the content of the response when it need not do so. This bloats those requests by 10-50% based on some upstream testing. Fixing this could reduce memory consumption substantially on expensive requests and it's fixed upstream in 3.4.16.

Version-Release number of selected component (if applicable):
OCP 4.6-4.8, etcd-3.4.15 and earlier

See upstream issue and PR for more analysis
https://github.com/etcd-io/etcd/issues/12835
https://github.com/etcd-io/etcd/pull/12871

Since we should likely just update to the latest, 3.4.18 and that version contains fixes present in 3.5.2 but not 3.5.0 currently utilized by OCP 4.9 I will mark this as being dependent on the existing bugs which track updating 4.9 to 3.5.3. This ensures that we don't update 4.6-4.8 and then regress by upgrading into 4.9 which lacks the same fixes.

The logging memory consumption issue is already fixed in etcd 3.5.

Comment 2 melbeher 2022-05-12 12:52:49 UTC

Etcd 3.5.3 has been bumped into 4.9 release branch 

Does this suffice for this bug @sdodson

Comment 3 Scott Dodson 2022-05-12 13:52:06 UTC

Unless there's specific reason not to I'd like to see 4.6 through 4.8 bumped to the latest 3.4.z, at least 3.4.16 with the cited fix.

If it's too much work to get all those versions then 4.8 will suffice given we see most 4.6 and 4.7 clusters quickly upgrading to 4.8.

Comment 4 melbeher 2022-05-12 14:03:21 UTC

Ok, I will start with 4.8 and update here