Bug 1956955

Summary:	Services sync causes too many ovn load balancer deletes
Product:	OpenShift Container Platform	Reporter:	Mohit Sheth <msheth>
Component:	Networking	Assignee:	Tim Rozet <trozet>
Networking sub component:	ovn-kubernetes	QA Contact:	Kedar Kulkarni <kkulkarn>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	high
Priority:	high	CC:	jlema, kkulkarn, mifiedle, sidsharm, trozet
Version:	4.8	Keywords:	Performance
Target Milestone:	---
Target Release:	4.9.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	perfscale-ovn
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1986573 (view as bug list)		Environment:
Last Closed:	2021-10-18 17:30:51 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1986573

Description Mohit Sheth 2021-05-04 18:17:37 UTC

Description of problem:
While running cluster-density at 250 node scale with 2000 projects, we see that there are 46000 nbctl commands and out of those 29000 are remove load_balancer.*vips

Version-Release number of selected component (if applicable):
ovs-vswitchd (Open vSwitch) 2.15.0

How reproducible:


Steps to Reproduce:
1. Run cluster-density 2k at 250 node scale

Actual results:
Lots of remove load_balancer deletes

Expected results:
Lesser deletes

Additional info:
ovnkube-master logs http://dell-r510-01.perf.lab.eng.rdu2.redhat.com/msheth/ovnkube-master-04-may/ovnkube-master-mkzfk.log

Comment 16 Kedar Kulkarni 2021-07-14 20:26:16 UTC

Hi,

I ran Cluster Density 250 workers for 2000 iterations and grepped through the log, the remove load_balancer call count was 15322, vs 29000 as reported originally. There were no errors there about bash too many args. I used the following build "4.9.0-0.nightly-2021-07-12-203753" . Based on this info, I am closing this bz as verified. 

Thanks,
KK.

Comment 19 errata-xmlrpc 2021-10-18 17:30:51 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:3759