Bug 1012143 - Conductor should be able to start multiple workers
Summary: Conductor should be able to start multiple workers
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 3.0
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: z3
: 3.0
Assignee: Xavier Queralt
QA Contact: Gabriel Szasz
URL:
Whiteboard:
: 1012012 1025352 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-09-25 20:04 UTC by Dan Smith
Modified: 2019-09-09 13:33 UTC (History)
9 users (show)

Fixed In Version: openstack-nova-2013.1.4-1.el6ost
Doc Type: Enhancement
Doc Text:
A new configuration option, called "workers", has been added to the conductor section in nova.conf, which specifies the number of workers the conductor will start. This was added to enhance performance on big clusters, where having just one conductor may cause performance issues due to only one service managing all the requests to the database.
Clone Of:
: 1012148 (view as bug list)
Environment:
Last Closed: 2013-11-18 15:16:20 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:1510 0 normal SHIPPED_LIVE Red Hat OpenStack 3.0 bug fix and enhancement advisory 2013-11-18 20:11:18 UTC

Description Dan Smith 2013-09-25 20:04:36 UTC
Description of problem:

Any reasonable deployment needs more than one conductor process and grizzly did not support this natively. Right now, this is only workable if someone runs a conductor on several machines or starts multiple processes manually. We should have an /etc/sysconfig/nova-conductor config point to specify how many daemons to start so that it's easy to start multiple daemons on a single system.

Version-Release number of selected component (if applicable):

openstack-nova-conductor-2013.1.3-3.el6ost

How reproducible:

Very.

Steps to Reproduce:
1. Deploy conductor on one machine
2. Deploy compute on many machines
3. Notice failures and poor performance

Actual results:

Everything funnels through a single conductor.

Expected results:

Multiple conductors should be started to spread out the load.

Additional info:

This should be pretty easy to do in the script, starting multiple workers, with log files and pid files like /var/log/nova/conductor-$i.log, etc.

Comment 7 Gabriel Szasz 2013-11-04 16:03:28 UTC
VERIFICATION:

1. Deploy conductor node and at least one compute node
2. Alter workers param of the [conductor] section of /etc/nova/nova.conf to
   workers=2
3. Restart the openstack-nova-conductor service
4. Check out the /var/log/nova/conductor.log

2013-11-04 10:57:49.387 28450 INFO nova.service [-] Starting 2 workers
2013-11-04 10:57:49.388 28450 INFO nova.service [-] Started child 28462
2013-11-04 10:57:49.390 28450 INFO nova.service [-] Started child 28463
2013-11-04 10:57:49.393 28462 AUDIT nova.service [-] Starting conductor node (version 2013.1.4-1.el6ost)
2013-11-04 10:57:49.394 28463 AUDIT nova.service [-] Starting conductor node (version 2013.1.4-1.el6ost)
2013-11-04 10:57:49.658 28463 INFO nova.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.16.96.136:5672
2013-11-04 10:57:49.658 28462 INFO nova.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.16.96.136:5672
2013-11-04 10:57:50.024 INFO nova.openstack.common.rpc.impl_qpid [req-54827805-a4f8-44a5-87a6-d646d68ffe58 None None] Connected to AMQP server on 10.16.96.136:5672
2013-11-04 10:57:52.975 INFO nova.openstack.common.rpc.impl_qpid [req-7bead73e-970d-4b41-b594-a61e3f23540c None None] Connected to AMQP server on 10.16.96.136:5672

5. The conductor node is indeed running 2 workers as set in nova.conf

Comment 8 errata-xmlrpc 2013-11-18 15:16:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1510.html

Comment 9 Dave Allan 2013-11-25 22:18:56 UTC
*** Bug 1012012 has been marked as a duplicate of this bug. ***

Comment 10 Dan Smith 2013-12-05 18:57:58 UTC
*** Bug 1025352 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.