Bug 2052975

Summary:	High cpu load on Juniper Qfx5120 Network switches after upgrade to Openshift 4.8.26
Product:	OpenShift Container Platform	Reporter:	Jonas Nordell <jnordell>
Component:	Networking	Assignee:	Surya Seetharaman <surya>
Networking sub component:	ovn-kubernetes	QA Contact:	huirwang
Status:	CLOSED ERRATA	Docs Contact:
Severity:	urgent
Priority:	high	CC:	anbhat, huirwang, mgokhool, openshift-bugs-escalate, surya
Version:	4.8	Keywords:	FastFix
Target Milestone:	---
Target Release:	4.11.0
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:
Clones:	2063834 (view as bug list)		Environment:
Last Closed:	2022-08-10 10:49:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2054394
Bug Blocks:	2063834

Description Jonas Nordell 2022-02-10 11:34:22 UTC

Description of problem:

After a cluster upgrade from OCP 4.7.33 to OCP 4.8.26 a big increase in CPU load on the network switches has started to affect all connected entities. The reason seams to be related to the fact that the switches is discovering Service IP addresses on the node NIC. 


Version-Release number of selected component (if applicable):

OCP 3.8.26


How reproducible:
In the customer environment on several clusters. 

Steps to Reproduce:
1. Upgrade cluster from 4.7.33 to 4.8.26
2. Look at assigned IPs on the network
3.

Actual results:
The switch is detecting many service IPs 

Expected results:
The switch should not detect service IPs

Additional info:

Comment 18 Surya Seetharaman 2022-02-14 21:13:31 UTC

Marking BZ with a depends on https://bugzilla.redhat.com/show_bug.cgi?id=2054394 since the fix really needs to go into OVN before we can consume it. This bug will continue to be open to track the OVN bump needed to consume this feature and to disable the GARPs for LB IPs on OVN-K side for egressIPs once the OVN fix lands. Note that this will take a few weeks.

Comment 22 Surya Seetharaman 2022-02-28 17:11:10 UTC

Upstream fix: https://github.com/ovn-org/ovn-kubernetes/pull/2835

Comment 24 Surya Seetharaman 2022-03-07 08:46:41 UTC

moving it back to post to pick up the second commit: https://github.com/ovn-org/ovn-kubernetes/pull/2835

Comment 30 errata-xmlrpc 2022-08-10 10:49:18 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069