Bug 1925586
Summary: | cluster-etcd-operator is leaking transports | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | Sam Batschelet <sbatsche> | ||||||||
Component: | Etcd | Assignee: | Sam Batschelet <sbatsche> | ||||||||
Status: | CLOSED ERRATA | QA Contact: | ge liu <geliu> | ||||||||
Severity: | high | Docs Contact: | |||||||||
Priority: | high | ||||||||||
Version: | 4.6 | ||||||||||
Target Milestone: | --- | ||||||||||
Target Release: | 4.8.0 | ||||||||||
Hardware: | Unspecified | ||||||||||
OS: | Unspecified | ||||||||||
Whiteboard: | |||||||||||
Fixed In Version: | Doc Type: | Bug Fix | |||||||||
Doc Text: |
Cause: Fix transport leak in etcd-operator
Consequence: The memory usage grows over time.
Fix: Free the memory allocated for transports to fix the memory leak
Result: The memory usage doesn't monotonically grow.
|
Story Points: | --- | ||||||||
Clone Of: | Environment: | ||||||||||
Last Closed: | 2021-07-27 22:41:35 UTC | Type: | Bug | ||||||||
Regression: | --- | Mount Type: | --- | ||||||||
Documentation: | --- | CRM: | |||||||||
Verified Versions: | Category: | --- | |||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||
Embargoed: | |||||||||||
Bug Depends On: | |||||||||||
Bug Blocks: | 1925739 | ||||||||||
Attachments: |
|
Sam, I keep eye on new payload which include the fix, but 4.7 have not prompt new build in this day, let's waiting for new payload to verify it, and I have reproduced it with current payload. Name Phase Started Failures Upgrades 4.7.0-0.nightly-2021-02-06-084550 Accepted 39 hours ago Created attachment 1755562 [details]
reproduce etcd operator memroy leaking
Verified with 4.8.0-0.ci-2021-02-06-233530, memory graph attached. Created attachment 1755817 [details]
48_memory_graph
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2438 |
Created attachment 1755250 [details] 4.7.0-0.nightly-2021-02-02-223803 steady state Description of problem: The operator metrics controller is currently not cleaning up transports the result is a leak. This leak also appears to bear a memory penalty. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. observe operator memory consumption over steady-state. 2. 3. Actual results: memory usage grows over time. Expected results: memory usage is reflective of $work Additional info: