Red Hat Bugzilla – Bug 1303803
rabbitmq resource-agents wipes too much data on recovery and kills additional users configuration
Last modified: 2016-11-03 20:01:28 EDT
Description of problem:
Ability to create RabbitMQ users that sustain across resource restarts. Justification is for monitoring capabilities that don't use the RabbitMQ admin user, and not using the guest account with its randomized password (i.e., create a specific monitor user with a specific password).
This should be a pretty simple fix if I understand what the problem
is. What happens currently in the resource agent is roughly:
- Start node
- If it's the only one up so far, then do nothing and wait for other
nodes. If at least one other node is already up, join cluster with
the other node.
- If the cluster join fails, then blow away mnesia (literally rm -rf
/var/lib/rabbitmq/mnesia/*) and try again. Unfortunately mnesia is
also where the user config is stored, so this is where the users get
So, we just need to be less heavy-handed when attempting to right the
cluster. Some combination of the rabbitmqctl commands
forget_cluster_node [--offline] / update_cluster_nodes / force_boot
should let us patch things up without completely removing mnesia and
all of the users with it.
Indeed we shouldn't remove so much data while restarting.
Ok, we almost there. Thanks to the recently built resource-agents-3.9.5-54.el7_2.7.x86_64 it's much easy to fix the cluster now. No need to wipe out Mnesia entirely.
I'm going to propose a fix soon.
I've got a patch which fixes that. I'm going to test it a little and provide a test build in a 4-6 hours.
This patch restores only "users" (rabbit_user) table. Other tables will be purged.
Created attachment 1131859 [details]
A proposed solution
tested according comment #7. User still exist after resource restart.
*** Bug 1297859 has been marked as a duplicate of this bug. ***
We use to add a monitoring user like:
rabbitmqctl add_user nagios PASSWORD
rabbitmqctl set_user_tags nagios monitoring
rabbitmqctl set_permissions nagios ".*" ".*" ".*"
Every time RabbitMQ is restarted, permissions are lost for this user.
PR to backup also this perms:
(In reply to adrianlzt from comment #19)
> We use to add a monitoring user like:
> rabbitmqctl add_user nagios PASSWORD
> rabbitmqctl set_user_tags nagios monitoring
> rabbitmqctl set_permissions nagios ".*" ".*" ".*"
> Every time RabbitMQ is restarted, permissions are lost for this user.
> PR to backup also this perms:
Thanks for the addition! I'll merge it shortly.
(In reply to Peter Lemenkov from comment #20)
> (In reply to adrianlzt from comment #19)
> > We use to add a monitoring user like:
> > rabbitmqctl add_user nagios PASSWORD
> > rabbitmqctl set_user_tags nagios monitoring
> > rabbitmqctl set_permissions nagios ".*" ".*" ".*"
> > Every time RabbitMQ is restarted, permissions are lost for this user.
> > PR to backup also this perms:
> > https://github.com/lemenkov/resource-agents/pull/1
> Thanks for the addition! I'll merge it shortly.
I didn't saw the code about userperm in:
Did we solved this problem there?
(In reply to xiaoxwan from comment #22)
> I didn't saw the code about userperm in:
> https://github.com/ClusterLabs/resource-agents/master .
> Did we solved this problem there?
No not yet.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.