Bug 2159631
| Summary: | ovn database containers consume more memory when upgrading ovs version to 2.17-18 or higher | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux Fast Datapath | Reporter: | zenghui.shi <zshi> |
| Component: | openvswitch2.17 | Assignee: | Timothy Redaelli <tredaelli> |
| Status: | CLOSED ERRATA | QA Contact: | Zhiqiang Fang <zfang> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | FDP 22.L | CC: | ctrautma, jhsiao, msantana, ralongi, tredaelli, zfang |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openvswitch2.17-2.17.0-84.el8fdp | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2023-03-15 12:41:03 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
zenghui.shi
2023-01-10 08:51:58 UTC
This is due to a change in bz#2040292, where we increased the max_ethports from 128 to 1024 and this is needed for some configurations on Mellanox cards. Is it really a problem that OVS increases memory? (In reply to Timothy Redaelli from comment #1) > This is due to a change in bz#2040292, where we increased the max_ethports > from 128 to 1024 and this is needed for some configurations on Mellanox > cards. > > Is it really a problem that OVS increases memory? The use case is microshift (openshift kubernetes for small form factor and edge computing) which promises to reserve as much system resource as possible for workload. The current minimum system memory requirement for microshift deployment is about 2G RAM, in which the network plugin (ovnk) consumes around 100M. It matters if ovn database container increase >10M (10% of 100M) and there are two ovn database containers in total. Some other info that might be helpful to understand the performance impact: we also did several other optimizations to reduce the memory footprint of OVS, such as CPUAffinity and --no-mlockall to openvswitch service. Is this max_ethports configurable? As below logs, "nbdb" and "sbdb" are ovn database containers. The memory usage of these two containers are significantly reduced after we changed the ovs version from 2.17 to 3.1. The nb from 27M down to 3M and sb from 29M down to 5M. [microshift@netqe6 ~]$ kubectl top pods -n openshift-ovn-kubernetes --containers POD NAME CPU(cores) MEMORY(bytes) ovnkube-master-xggj5 nbdb 1m 27Mi <----------------- 27M ovnkube-master-xggj5 northd 1m 5Mi ovnkube-master-xggj5 ovnkube-master 3m 61Mi ovnkube-master-xggj5 sbdb 1m 29Mi <----------------- 29M ovnkube-node-5cj2n ovn-controller 1m 4Mi [microshift@netqe6 ~]$ oc -n openshift-ovn-kubernetes exec -it ovnkube-master-xggj5 -c northd -- bash [root@netqe6 ~]# rpm -qa | grep openvswitch openvswitch-selinux-extra-policy-1.0-29.el8fdp.noarch openvswitch2.17-2.17.0-62.el8fdp.x86_64 python3-openvswitch2.17-2.17.0-62.el8fdp.x86_64 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [microshift@netqe6 ~]$ kubectl top pods -n openshift-ovn-kubernetes --containers POD NAME CPU(cores) MEMORY(bytes) ovnkube-master-d9bbw nbdb 1m 3Mi <----------------- 3M ovnkube-master-d9bbw northd 1m 5Mi ovnkube-master-d9bbw ovnkube-master 2m 49Mi ovnkube-master-d9bbw sbdb 1m 5Mi <----------------- 5M ovnkube-node-d9s9r ovn-controller 1m 4Mi [microshift@netqe6 ~]$ [microshift@netqe6 ~]$ oc -n openshift-ovn-kubernetes exec -it ovnkube-master-d9bbw -c northd -- bash [root@netqe6 ~]# [root@netqe6 ~]# rpm -qa | grep openvswitch openvswitch-selinux-extra-policy-1.0-29.el8fdp.noarch openvswitch3.1-3.1.0-3.el8fdp.x86_64 python3-openvswitch3.1-3.1.0-3.el8fdp.x86_64 [root@netqe6 ~]# Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (openvswitch3.1 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2023:1256 * Mon Mar 27 2023 Open vSwitch CI <ovs-ci> - 2.17.0-84
- Merging upstream branch-2.17 [RH git: 2ae164f82b]
Commit list:
d6d1cad6a7 dpif-netlink: Always create at least 1 handler.
* Mon Mar 20 2023 Timothy Redaelli <tredaelli> - 2.17.0-83
- Install shared linked binaries (all, but ovs-vswitchd) [RH git: 57f5be87e0] (#2159631)
Resolves: #2159631
|