Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2011178

Summary:	networking-ovn high RSS memory usage
Product:	Red Hat OpenStack	Reporter:	anil venkata <vkommadi>
Component:	python-networking-ovn	Assignee:	Lucas Alvares Gomes <lmartins>
Status:	CLOSED ERRATA	QA Contact:	Eran Kuris <ekuris>
Severity:	high	Docs Contact:
Priority:	high
Version:	16.2 (Train)	CC:	apevec, dalvarez, jlibosva, jraju, lhh, majopela, scohen
Target Milestone:	z2	Keywords:	Scale, TestBlocker, Triaged
Target Release:	16.2 (Train on RHEL 8.4)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	python-networking-ovn-7.4.2-2.20220113214852.a2eba10.el8ost	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:
Clones:	2126059 (view as bug list)		Environment:
Last Closed:	2022-03-23 22:11:48 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2126059

Comment 2 anil venkata 2021-10-10 08:09:18 UTC

We have a great improvement (i.e RSS memory reduced from 73.5Gib to 8.35Gib) with @Daniel Alvarez Sanchez  mac binding patch [1]
When we have 1.18 million MAC binding entries for 4k VMs, 2.4k floating ips and 800 routers, without the patch neutron-server is consuming 73.5Gib.  With the Daniel's patch [1] neutron-server is taking 8.35Gib.
No performance impact with the patch i.e both the tests have taken 8 hours 30 minutes to complete.

Grafana links for the testing without the fix [1]
RSS memory

https://snapshot.raintank.io/dashboard/snapshot/z7znO1rASYyJzbocjPXFTI9x4V5eTF1F

OSP DB resources during the testing

https://snapshot.raintank.io/dashboard/snapshot/rCmGlIjKahDGDS8S2DPRUElsQ4zqu2KZ

OVN DB resources without MAC_Binding and logical flows (note: collectd is calling ovsdb-client for these metrics)

https://snapshot.raintank.io/dashboard/snapshot/mcy703BkDVALyaLGzYqf1jkJ56U8rpLR

MAC_Binding and logical_flows count (note: collectd is calling ovsdb-client for these metrics)

https://snapshot.raintank.io/dashboard/snapshot/SewHBjAZDoW0tuG4N7N79nUw8xMWAN1D




Grafana links for the testing with the mac binding fix [1]
MAC binding fix RSS memory

https://snapshot.raintank.io/dashboard/snapshot/pbXcD0kABoJ8L4BX7MU8x41RJfJMORtj

OSP DB resources during the testing

https://snapshot.raintank.io/dashboard/snapshot/VKZ75F20L9uuO8YSXD6XkKMzuwXj8pz4

OVN DB resources without MAC_Binding and logical flows (note: collectd is calling ovsdb-client for these metrics)

https://snapshot.raintank.io/dashboard/snapshot/a9V1zcDxiM0B62bq86CsFcj0SE323WiQ

MAC_Binding and logical_flows count (note: collectd is calling ovsdb-client for these metrics)

https://snapshot.raintank.io/dashboard/snapshot/4I46csupL1WCqVINLM66nTVeeUgt9sBg

Comment 9 anil venkata 2022-02-22 09:24:32 UTC

We used the puddle RHOS-16.2-RHEL-8-20220201.n.1 in our scale testing and we could see neutron server using 8G RSS memory for 4K VMs.
SO the fix is working as expected.

Comment 10 Eran Kuris 2022-02-22 09:27:22 UTC

Following comment 9 I am setting this bz to verified 
 thanks a lot, Anil for helping with that issue.

Comment 15 errata-xmlrpc 2022-03-23 22:11:48 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 16.2.2), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1001