Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1427330

Summary: performance degradation in agent<->server port wiring process due to slow queries?
Product: Red Hat OpenStack Reporter: Benjamin Schmaus <bschmaus>
Component: openstack-neutronAssignee: Terry Wilson <twilson>
Status: CLOSED ERRATA QA Contact: Alexander Stafeyev <astafeye>
Severity: high Docs Contact:
Priority: high    
Version: 10.0 (Newton)CC: amuller, bschmaus, chrisw, jlibosva, mbayer, nyechiel, oblaut, srevivo, twilson
Target Milestone: z3Keywords: Triaged, ZStream
Target Release: 10.0 (Newton)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-neutron-9.2.0-3.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-06-28 15:31:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Benjamin Schmaus 2017-02-27 22:56:01 UTC
Description of problem:
Slow performance when creating a neutron port due to slow queries

http://lists.openstack.org/pipermail/openstack-dev/2017-February/112266.html

Version-Release number of selected component (if applicable):
OSP10

How reproducible:
100%

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Benjamin Schmaus 2017-02-27 22:57:03 UTC
Additional link: https://review.openstack.org/#/c/430973/

Comment 2 Michael Bayer 2017-03-03 18:45:30 UTC
So, as noted a lot of this slowness is related to SQLAlchemy queries, and along with my work with Daniel I have also been working with a different group of Neutron users to build in performance enhancements into SQLAlchemy 1.1, as a result of observed latencies that have accumulated since the 0.8 series.  The issue describing this work is at https://bitbucket.org/zzzeek/sqlalchemy/issues/3915/performance-degradation-on-version-10xx and these fixes are now available in SQLAlchemy 1.1.6.  They are targeted at the particular kind of query that Neutron uses which includes a large amount of "eager joins".    This does not reduce the SQL overhead but reduces the Python overhead associated with the many SQL queries by about 40%.

Additionally however, Neutron itself in very recent patches has also been working on this same area by reworking the queries, and that work can be seen in a review such as https://review.openstack.org/#/c/408143/, here they are changing the type of query used in many cases, though I'm not certain if the impact of these changes has been measured, or if these changes are just another way of working around the SQLAlchemy slowness that has been largely improved.

I would encourage the people working on this customer issue to try out SQLAlchemy 1.1.6 with Neutron to see if this improves performance or not.  Openstack upstream is still pinned at the 1.0.x series of SQLAlchemy, but if we wanted these improvements ported into rhos 10 there are various avenues for that (I can backport them into 1.0.x upstream, we can patch them into 1.0.x at the specfile level, or we can push to use SQLAlchemy 1.1).

Comment 3 Michael Bayer 2017-03-03 19:17:55 UTC
Here's an upstream backport of the SQLAlchemy changes to the 1.0.x series: https://gerrit.sqlalchemy.org/#/c/325/

Comment 16 errata-xmlrpc 2017-06-28 15:31:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1594