Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1347371 - RFE: Balance Rabbitmq Queue Master Location on queue declaration with min-masters strategy
RFE: Balance Rabbitmq Queue Master Location on queue declaration with min-mas...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates (Show other bugs)
10.0 (Newton)
Unspecified Unspecified
unspecified Severity medium
: rc
: 10.0 (Newton)
Assigned To: Michele Baldessari
Marian Krcmarik
: FutureFeature, InstallerIntegration, Triaged
Depends On:
Blocks: 1384167
  Show dependency treegraph
 
Reported: 2016-06-16 11:53 EDT by Marian Krcmarik
Modified: 2016-12-14 10:40 EST (History)
14 users (show)

See Also:
Fixed In Version: openstack-tripleo-heat-templates-5.0.0-0.20161008015357.0d3e3e3.1.el7ost
Doc Type: Enhancement
Doc Text:
With this enhancement, RabbitMQ introduces the new HA feature of Queue Master distribution. One of the strategies is `min-masters`, which picks the node hosting the minimum number of masters. This was added because of the possibility that one of the controllers may become unavailable, with Queue Masters then located on available controllers during queue declarations. Once the lost controller becomes available again, masters of newly-declared queues are not placed with priority to the controller with an obviously lower number of queue masters, and consequently the distribution may be unbalanced, with one of the controllers under significantly higher load in the event of multiple fail-overs. As a result, this enhancement spreads out the queues across controllers after a controller fail-over.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-12-14 10:40:14 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1629010 None None None 2016-09-29 12:43 EDT
OpenStack gerrit 379596 None None None 2016-09-29 12:52 EDT
Red Hat Product Errata RHEA-2016:2948 normal SHIPPED_LIVE Red Hat OpenStack Platform 10 enhancement update 2016-12-14 14:55:27 EST

  None (edit)
Description Marian Krcmarik 2016-06-16 11:53:41 EDT
Description of problem:
It may happen that one of the controllers may become unvailable and Queue Masters will be located on available controllers during queue declarations. Once lost controller will be become available masters of newly declared queues are not placed  prioritly to such controller with obviously lower number of queue masters and thus the distribution may be unbalanced and one of the controllers may become under significantly higher load in some circumstainces of multiple failovers.

rabbitmq introduced a new HA feature of Queue masters distribution - one of the strategies is min-masters, which picks the node hosting the minimum number of masters.

One of the ways how to turn such min-masters strategy on is by adding following into configuration file - rabbitmq.config
{rabbit,[ .
          .
          {queue_master_locator, <<"min-masters">>},
          .
          . ]},

The other two strategies are Client-Local (default) and Random.

Please consider turning min-masters strategy on in rabbitmq configuration.
Comment 2 Andrew Beekhof 2016-06-28 19:36:42 EDT
The goal of balancing the queues seems like a good one to me.

Chris, can you take care of updating director after validating with Peter that this is the best way to achieve it please?  I'd not expect this option to decrease availability in any way, but best to be sure.
Comment 6 Marian Krcmarik 2016-10-19 16:41:14 EDT
Verified.

$ sudo grep min-masters /etc/rabbitmq/rabbitmq.config
    {queue_master_locator, <<"min-masters">>},
Comment 8 errata-xmlrpc 2016-12-14 10:40:14 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html

Note You need to log in before you can comment on or make changes to this bug.