Bug 1012143 - Conductor should be able to start multiple workers
Conductor should be able to start multiple workers
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova (Show other bugs)
3.0
All Linux
unspecified Severity medium
: z3
: 3.0
Assigned To: Xavier Queralt
Gabriel Szasz
: ZStream
: 1012012 1025352 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-09-25 16:04 EDT by Dan Smith
Modified: 2014-06-18 03:00 EDT (History)
10 users (show)

See Also:
Fixed In Version: openstack-nova-2013.1.4-1.el6ost
Doc Type: Enhancement
Doc Text:
A new configuration option, called "workers", has been added to the conductor section in nova.conf, which specifies the number of workers the conductor will start. This was added to enhance performance on big clusters, where having just one conductor may cause performance issues due to only one service managing all the requests to the database.
Story Points: ---
Clone Of:
: 1012148 (view as bug list)
Environment:
Last Closed: 2013-11-18 10:16:20 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Dan Smith 2013-09-25 16:04:36 EDT
Description of problem:

Any reasonable deployment needs more than one conductor process and grizzly did not support this natively. Right now, this is only workable if someone runs a conductor on several machines or starts multiple processes manually. We should have an /etc/sysconfig/nova-conductor config point to specify how many daemons to start so that it's easy to start multiple daemons on a single system.

Version-Release number of selected component (if applicable):

openstack-nova-conductor-2013.1.3-3.el6ost

How reproducible:

Very.

Steps to Reproduce:
1. Deploy conductor on one machine
2. Deploy compute on many machines
3. Notice failures and poor performance

Actual results:

Everything funnels through a single conductor.

Expected results:

Multiple conductors should be started to spread out the load.

Additional info:

This should be pretty easy to do in the script, starting multiple workers, with log files and pid files like /var/log/nova/conductor-$i.log, etc.
Comment 7 Gabriel Szasz 2013-11-04 11:03:28 EST
VERIFICATION:

1. Deploy conductor node and at least one compute node
2. Alter workers param of the [conductor] section of /etc/nova/nova.conf to
   workers=2
3. Restart the openstack-nova-conductor service
4. Check out the /var/log/nova/conductor.log

2013-11-04 10:57:49.387 28450 INFO nova.service [-] Starting 2 workers
2013-11-04 10:57:49.388 28450 INFO nova.service [-] Started child 28462
2013-11-04 10:57:49.390 28450 INFO nova.service [-] Started child 28463
2013-11-04 10:57:49.393 28462 AUDIT nova.service [-] Starting conductor node (version 2013.1.4-1.el6ost)
2013-11-04 10:57:49.394 28463 AUDIT nova.service [-] Starting conductor node (version 2013.1.4-1.el6ost)
2013-11-04 10:57:49.658 28463 INFO nova.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.16.96.136:5672
2013-11-04 10:57:49.658 28462 INFO nova.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.16.96.136:5672
2013-11-04 10:57:50.024 INFO nova.openstack.common.rpc.impl_qpid [req-54827805-a4f8-44a5-87a6-d646d68ffe58 None None] Connected to AMQP server on 10.16.96.136:5672
2013-11-04 10:57:52.975 INFO nova.openstack.common.rpc.impl_qpid [req-7bead73e-970d-4b41-b594-a61e3f23540c None None] Connected to AMQP server on 10.16.96.136:5672

5. The conductor node is indeed running 2 workers as set in nova.conf
Comment 8 errata-xmlrpc 2013-11-18 10:16:20 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1510.html
Comment 9 Dave Allan 2013-11-25 17:18:56 EST
*** Bug 1012012 has been marked as a duplicate of this bug. ***
Comment 10 Dan Smith 2013-12-05 13:57:58 EST
*** Bug 1025352 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.