Bug 1466019 - Seeing DBDeadlock on neutron server log when attempting to create networks and subnets
Seeing DBDeadlock on neutron server log when attempting to create networks an...
Status: NEW
Product: Red Hat OpenStack
Classification: Red Hat
Component: puppet-mysql (Show other bugs)
10.0 (Newton)
Unspecified Unspecified
unspecified Severity medium
: ---
: ---
Assigned To: RHOS Maint
nlevinki
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-28 13:43 EDT by bigswitch
Modified: 2017-06-28 14:20 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description bigswitch 2017-06-28 13:43:49 EDT
Description of problem:
We are trying to scale RHOSP 10 with 900 projects, one logical router per project, and four network with one subnet each per tenant with a script.
The setup has three controller and one compute node deployed. Each controller has 48 cores and 380gig ram.
The script creates 900 tenants first, than attempt to create 900 logical routers and fail at about 150 logical routers. It than attempts to create 4 networks per tenant and at about 400 networks/subnets it is seeing 504 gateway timeout error. There is also DBDeadlock error in neutron-server log.
Would like to know if there is any settings we can tweak to improve the database performance.

Version-Release number of selected component (if applicable):
RHOSP 10

How reproducible:
Always

Will upload sosreport from three controllers to box.
Comment 1 bigswitch 2017-06-28 13:45:22 EDT
The sosreport is uploaded to box

https://bigswitch.box.com/s/ujvfkhxs7gpsuirr09sujub5w9714nl2

Note You need to log in before you can comment on or make changes to this bug.