Bug 2078545
| Summary: | many KubeVirtComponentExceedsRequestedMemory alerts in Firing state | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Container Native Virtualization (CNV) | Reporter: | Denys Shchedrivyi <dshchedr> | ||||
| Component: | Virtualization | Assignee: | Jed Lejosne <jlejosne> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Akriti Gupta <akrgupta> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | urgent | ||||||
| Version: | 4.11.0 | CC: | acardace, kbidarka, kgordeev, mam, sgott, usurse | ||||
| Target Milestone: | --- | ||||||
| Target Release: | 4.11.4 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | hco-bundle-registry-container-v4.11.3-1 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 2151170 2166385 (view as bug list) | Environment: | |||||
| Last Closed: | 2023-05-30 15:37:44 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 2151170, 2166385 | ||||||
| Attachments: |
|
||||||
Denys, Is this issue seen recurring with the latest 4.11 builds ? yes, it is firing for some pods right after installation in v4.11.0-327 Raising the priority and severity of this because this can actually be a real problem on nodes with memory pressure. KubeVirt resources will be high on the list to be killed in such a case. The github PR linked with this issue was the one against the main branch. I just added the (un-merged) backport PR to release-0.53 that will affect CNV 4.11 Moving back to 4.11.1 as all components but virt-handler have been addressed and virt-handler itself is tracked specifically by https://bugzilla.redhat.com/show_bug.cgi?id=2134123. Moving this BZ to Assigned state since in my opinion all memory requests should be adjusted. After some research I see all of virt components numbers are exceed the limits (or very close to it) on CNV v4.11: virt-api - 245Mb virt-controller - 270Mb virt-handler - 248Mb virt-operator - 255Mb (There is Bug 2132473 with virt-operator which increases memory usage on every Kubevirt CR update, so it is difficult to predict how much memory it consumes by default) virt-controller memory request still has to be adjusted, deferring to 4.11.4 as we're past code freeze for 4.11.3. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Virtualization 4.11.4 Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2023:3352 |
Created attachment 1874828 [details] alerts Description of problem: There are many alerts KubeVirtComponentExceedsRequestedMemory in Firing state on a cluster, some of them appeared right after installation. May be memory should be adjusted for kubevirt system pods? Version-Release number of selected component (if applicable): 4.11 Actual results: Alerts are constantly firing, some of them triggered right after cluster installation. Screenshot attached Expected results: kubevirt system pods requested enough memory and alert is not triggered