Description of problem: It may happen that one of the controllers may become unvailable and Queue Masters will be located on available controllers during queue declarations. Once lost controller will be become available masters of newly declared queues are not placed prioritly to such controller with obviously lower number of queue masters and thus the distribution may be unbalanced and one of the controllers may become under significantly higher load in some circumstainces of multiple failovers. rabbitmq introduced a new HA feature of Queue masters distribution - one of the strategies is min-masters, which picks the node hosting the minimum number of masters. One of the ways how to turn such min-masters strategy on is by adding following into configuration file - rabbitmq.config {rabbit,[ . . {queue_master_locator, <<"min-masters">>}, . . ]}, The other two strategies are Client-Local (default) and Random. Please consider turning min-masters strategy on in rabbitmq configuration.
The goal of balancing the queues seems like a good one to me. Chris, can you take care of updating director after validating with Peter that this is the best way to achieve it please? I'd not expect this option to decrease availability in any way, but best to be sure.
Verified. $ sudo grep min-masters /etc/rabbitmq/rabbitmq.config {queue_master_locator, <<"min-masters">>},
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-2948.html