Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2024690

Summary:	Concurrent bulk requests make neutron-server use 100% CPU on all the workers
Product:	Red Hat OpenStack	Reporter:	Michał Dulko <mdulko>
Component:	openstack-neutron	Assignee:	Slawek Kaplonski <skaplons>
Status:	CLOSED ERRATA	QA Contact:	Candido Campos <ccamposr>
Severity:	high	Docs Contact:
Priority:	high
Version:	16.2 (Train)	CC:	ccamposr, chrisw, mdemaced, rcernin, scohen
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-neutron-15.3.5-2.20220113150030.94e4cbb.el8ost	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:
Clones:	2035302 (view as bug list)		Environment:
Last Closed:	2022-03-23 22:12:24 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2035302

Description Michał Dulko 2021-11-18 17:00:03 UTC

Description of problem:
We've discovered this while trying to work out scale tests with Kuryr. When creating more pods in OpenShift Kuryr will shoot a bunch of bulk port create requests. We've see issues with attempts to create ~300 ports in bulks of 10 in 10 subnets.

Version-Release number of selected component (if applicable):
16.2_20210811.1, standalone (I know, it's not supported, but I still think some issue is there)

How reproducible:
Pretty much always.

Steps to Reproduce:
1. Install OpenShift with Kuryr on top of OpenStack.
2. Create 10 namespaces with 10 pods in each.
3. Observe neutron-server behavior, start seeing 504 responses to port POST requests sent to Neutron by kuryr-controller.

I'm trying to find a smaller reproducer.

Actual results:
Neutron stops to be responsive.

Expected results:
Neutron survive.

Additional info:
I'm happy to provide a cluster for debugging this.

Comment 16 errata-xmlrpc 2022-03-23 22:12:24 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 16.2.2), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1001