Bug 1956955

Summary: Services sync causes too many ovn load balancer deletes
Product: OpenShift Container Platform Reporter: Mohit Sheth <msheth>
Component: NetworkingAssignee: Tim Rozet <trozet>
Networking sub component: ovn-kubernetes QA Contact: Kedar Kulkarni <kkulkarn>
Status: ASSIGNED --- Docs Contact:
Severity: high    
Priority: high CC: jlema, kkulkarn, mifiedle
Version: 4.8Keywords: Performance
Target Milestone: ---   
Target Release: 4.8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: perfscale-ovn
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Mohit Sheth 2021-05-04 18:17:37 UTC
Description of problem:
While running cluster-density at 250 node scale with 2000 projects, we see that there are 46000 nbctl commands and out of those 29000 are remove load_balancer.*vips

Version-Release number of selected component (if applicable):
ovs-vswitchd (Open vSwitch) 2.15.0

How reproducible:

Steps to Reproduce:
1. Run cluster-density 2k at 250 node scale

Actual results:
Lots of remove load_balancer deletes

Expected results:
Lesser deletes

Additional info:
ovnkube-master logs http://dell-r510-01.perf.lab.eng.rdu2.redhat.com/msheth/ovnkube-master-04-may/ovnkube-master-mkzfk.log