Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 2159631

Summary: ovn database containers consume more memory when upgrading ovs version to 2.17-18 or higher
Product: Red Hat Enterprise Linux Fast Datapath Reporter: zenghui.shi <zshi>
Component: openvswitch2.17Assignee: Timothy Redaelli <tredaelli>
Status: CLOSED ERRATA QA Contact: Zhiqiang Fang <zfang>
Severity: high Docs Contact:
Priority: unspecified    
Version: FDP 22.LCC: ctrautma, jhsiao, msantana, ralongi, tredaelli, zfang
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openvswitch2.17-2.17.0-84.el8fdp Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-03-15 12:41:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description zenghui.shi 2023-01-10 08:51:58 UTC
Description of problem:


When deploying ovn-kubernetes on microshift inside VM, it is found that there is a memory jump (<10M to >15M) on ovn-kubernetes database containers when upgrading ovs version from 2.17.0-8 to 2.17.0-18 in ovn-kubernetes dockerfile. The issue also exists with latest ovs version 2.17.0-62.

The container memory are observed with metrics-server: https://github.com/kubernetes-sigs/metrics-server

VM spec: 4 vCPUs, 4Gi memory

Version-Release number of selected component (if applicable):

downstream ovn-kubernetes: dbdf0372b2f2be0c3413731eeae3084d924878b6
ovn: 22.09.0-25
ovs: ovs-2.17.0-62



Also run the same tests with different ovn/ovs versions:

ovn 22.09.0-25/ovs-2.17.0-8: low memory
ovn 22.09.0-25/ovs-2.17.0-18: high memory
ovn 22.09.0-25/ovs-2.17.0-22: high memory
ovn 22.09.0-25/ovs-2.17.0-50: high memory
ovn 22.09.0-25/ovs-2.17.0-62: high memory

ovn 22.09.0-22/ovs-2.17.0-8: low memory
ovn 22.09.0-11/ovs-2.17.0-8: low memory
ovn 22.09.0-10/ovs-2.17.0-8: low memory
ovn 22.09.0-5/ovs-2.17.0-8: low memory
ovn 22.09.0-4/ovs-2.17.0-8: low memory
ovn 22.06.0-82/ovs-2.17.0-8: low memory
ovn 22.06.0-preview.branched.38/ovs-2.17.0-8: low memory
ovn 22.09.0-25/ovs-2.17.0-62: high memory

Comment 1 Timothy Redaelli 2023-01-31 17:44:23 UTC
This is due to a change in bz#2040292, where we increased the max_ethports from 128 to 1024 and this is needed for some configurations on Mellanox cards.

Is it really a problem that OVS increases memory?

Comment 2 zenghui.shi 2023-02-01 05:30:12 UTC
(In reply to Timothy Redaelli from comment #1)
> This is due to a change in bz#2040292, where we increased the max_ethports
> from 128 to 1024 and this is needed for some configurations on Mellanox
> cards.
> 
> Is it really a problem that OVS increases memory?

The use case is microshift (openshift kubernetes for small form factor and edge computing) which promises to reserve as much system resource as possible for workload.
The current minimum system memory requirement for microshift deployment is about 2G RAM, in which the network plugin (ovnk) consumes around 100M. It matters if ovn database container increase >10M (10% of 100M) and there are two ovn database containers in total.

Some other info that might be helpful to understand the performance impact: we also did several other optimizations to reduce the memory footprint of OVS, such as CPUAffinity and --no-mlockall to openvswitch service.

Is this max_ethports configurable?

Comment 8 Zhiqiang Fang 2023-03-02 18:39:14 UTC
As below logs, "nbdb" and "sbdb" are ovn database containers. The memory usage of these two containers are significantly reduced after we changed the ovs version from 2.17 to 3.1. The nb from 27M down to 3M and sb from 29M down to 5M.

[microshift@netqe6 ~]$ kubectl top  pods -n openshift-ovn-kubernetes --containers
POD                    NAME             CPU(cores)   MEMORY(bytes)   
ovnkube-master-xggj5   nbdb             1m           27Mi  <----------------- 27M      
ovnkube-master-xggj5   northd           1m           5Mi             
ovnkube-master-xggj5   ovnkube-master   3m           61Mi            
ovnkube-master-xggj5   sbdb             1m           29Mi  <----------------- 29M       
ovnkube-node-5cj2n     ovn-controller   1m           4Mi  

[microshift@netqe6 ~]$ oc -n openshift-ovn-kubernetes exec -it ovnkube-master-xggj5 -c northd -- bash
[root@netqe6 ~]#  rpm -qa | grep openvswitch
openvswitch-selinux-extra-policy-1.0-29.el8fdp.noarch
openvswitch2.17-2.17.0-62.el8fdp.x86_64
python3-openvswitch2.17-2.17.0-62.el8fdp.x86_64

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[microshift@netqe6 ~]$ kubectl top  pods -n openshift-ovn-kubernetes --containers
POD                    NAME             CPU(cores)   MEMORY(bytes)   
ovnkube-master-d9bbw   nbdb             1m           3Mi   <----------------- 3M    
ovnkube-master-d9bbw   northd           1m           5Mi             
ovnkube-master-d9bbw   ovnkube-master   2m           49Mi            
ovnkube-master-d9bbw   sbdb             1m           5Mi   <----------------- 5M   
ovnkube-node-d9s9r     ovn-controller   1m           4Mi             
[microshift@netqe6 ~]$ 

[microshift@netqe6 ~]$ oc -n openshift-ovn-kubernetes exec -it ovnkube-master-d9bbw -c northd -- bash
[root@netqe6 ~]# 
[root@netqe6 ~]# rpm -qa | grep openvswitch
openvswitch-selinux-extra-policy-1.0-29.el8fdp.noarch
openvswitch3.1-3.1.0-3.el8fdp.x86_64
python3-openvswitch3.1-3.1.0-3.el8fdp.x86_64
[root@netqe6 ~]#

Comment 11 errata-xmlrpc 2023-03-15 12:41:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (openvswitch3.1 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:1256

Comment 12 OvS team 2023-03-28 03:39:45 UTC
* Mon Mar 27 2023 Open vSwitch CI <ovs-ci> - 2.17.0-84
- Merging upstream branch-2.17 [RH git: 2ae164f82b]
    Commit list:
    d6d1cad6a7 dpif-netlink: Always create at least 1 handler.


* Mon Mar 20 2023 Timothy Redaelli <tredaelli> - 2.17.0-83
- Install shared linked binaries (all, but ovs-vswitchd) [RH git: 57f5be87e0] (#2159631)
    Resolves: #2159631