Back to bug 1381620

Who When What Removed Added
anil venkata 2016-10-04 15:01:54 UTC Target Release 7.0 (Kilo) 10.0 (Newton)
Assaf Muller 2016-10-04 16:31:35 UTC Assignee amuller vkommadi
Assaf Muller 2016-10-04 16:31:55 UTC Status NEW ASSIGNED
Jon Schlueter 2016-10-14 12:50:08 UTC Status ASSIGNED MODIFIED
CC jschluet
Fixed In Version openstack-neutron-9.0.0-1.2.el7ost
Jon Schlueter 2016-11-18 02:36:50 UTC Keywords ZStream Triaged
Target Milestone async rc
errata-xmlrpc 2016-11-18 02:58:18 UTC Status MODIFIED ON_QA
Toni Freger 2016-11-20 12:00:14 UTC CC vkommadi
Flags needinfo?(vkommadi)
anil venkata 2016-11-21 09:13:43 UTC Flags needinfo?(vkommadi)
John Schwarz 2016-11-22 06:55:52 UTC CC jschwarz
Toni Freger 2016-11-22 09:11:34 UTC QA Contact tfreger astafeye
Alexander Stafeyev 2016-11-23 13:10:00 UTC Flags needinfo?(vkommadi)
anil venkata 2016-11-23 13:49:42 UTC Flags needinfo?(vkommadi)
Alexander Stafeyev 2016-11-23 14:14:34 UTC Status ON_QA VERIFIED
anil venkata 2016-12-06 06:52:09 UTC Doc Text Cause:
Currently max number of client connections(i.e greenlets spawned at a time) opened at any time by the WSGI server is set to 100 with wsgi_default_pool_size[1].

Consequence:
This configuration may be fine for neutron api server. But with wsgi_default_pool_size(=100) requests, state change server is creating heavy cpu load on l3 agent, causing l3 agent to crash.

Fix:
So this server(which run on agents) need lesser value i.e can be configured to half the number of cpu on agent.
New config option "ha_keepalived_state_change_server_threads" added to configure number of threads in state change server instead of wsgi_default_pool_size.

Result:
l3 agent not crashing after spawning many state change server threads with this config option.
Doc Type If docs needed, set a value Bug Fix
Don Domingo 2016-12-14 02:00:01 UTC CC ddomingo
Doc Text Cause:
Currently max number of client connections(i.e greenlets spawned at a time) opened at any time by the WSGI server is set to 100 with wsgi_default_pool_size[1].

Consequence:
This configuration may be fine for neutron api server. But with wsgi_default_pool_size(=100) requests, state change server is creating heavy cpu load on l3 agent, causing l3 agent to crash.

Fix:
So this server(which run on agents) need lesser value i.e can be configured to half the number of cpu on agent.
New config option "ha_keepalived_state_change_server_threads" added to configure number of threads in state change server instead of wsgi_default_pool_size.

Result:
l3 agent not crashing after spawning many state change server threads with this config option.
Previously, the maximum number of client connections (i.e greenlets spawned at a time) opened at any time by the WSGI server was set to 100 with 'wsgi_default_pool_size'. While this setting was adequate for the OpenStack Networking API server, the state change server created heavy CPU loads on the L3 agent, which caused the agent to crash.

With this release, you can now use the new 'ha_keepalived_state_change_server_threads' setting to configure the number of threads in the state change server. Client connections are no longer limited by 'wsgi_default_pool_size', thereby avoiding an L3 agent crash when many state change server threads are spawned.
errata-xmlrpc 2016-12-14 13:47:58 UTC Status VERIFIED RELEASE_PENDING
errata-xmlrpc 2016-12-14 16:07:37 UTC Status RELEASE_PENDING CLOSED
Resolution --- ERRATA
Last Closed 2016-12-14 11:07:37 UTC

Back to bug 1381620