Red Hat Bugzilla – Bug 240508
2node flag doesn't clear properly
Last modified: 2009-04-16 18:30:05 EDT
Description of problem: When a third node is added to a formerly 2node-mode cluster, cman fails to clear the 2node flag. The cluster is functional (failover of a service to/from the third node still works fine) but the only way to clear the flag on nodes 1 & 2 is to reboot them. Version-Release number of selected component (if applicable): cman-2.0.60-1.el5 How reproducible: Every time. Steps to Reproduce: 1. Start with functional 2-node failover cluster running (web) service. 2. Use s-c-c to add a 3rd node, then Send to Cluster. Manual copy the cluster.conf to 3rd node. 3. node-3# service cman start; service rgmanager start Actual results: After node-3 joins cluster, output of "cman_tool status" on node-1 and node-3, and output of "cman_tool nodes" on node-1, respectively: [root@node-1 ~]# cman_tool status Version: 6.0.1 Config Version: 2 Cluster Name: cluster7 Cluster Id: 26783 Cluster Member: Yes Cluster Generation: 12 Membership state: Cluster-Member Nodes: 3 Expected votes: 1 Total votes: 3 Quorum: 2 Active subsystems: 7 Flags: 2node Ports Bound: 0 177 Node name: node-1.cluster-7.example.com Node ID: 1 Multicast addresses: 239.192.104.8 Node addresses: 172.16.36.71 [root@node-3 ~]# cman_tool status Version: 6.0.1 Config Version: 2 Cluster Name: cluster7 Cluster Id: 26783 Cluster Member: Yes Cluster Generation: 12 Membership state: Cluster-Member Nodes: 3 Expected votes: 3 Total votes: 3 Quorum: 2 Active subsystems: 7 Flags: Ports Bound: 0 177 Node name: node-3.cluster-7.example.com Node ID: 3 Multicast addresses: 239.192.104.8 Node addresses: 172.16.36.73 [root@node-1 ~]# cman_tool nodes Node Sts Inc Joined Name 1 M 4 2007-05-17 19:15:37 node-1.cluster-7.example.com 2 M 8 2007-05-17 19:15:37 node-2.cluster-7.example.com 3 M 12 2007-05-17 19:35:11 node-3.cluster-7.example.com Expected results: When the third node joined the cluster, nodes 1 and 2 should have had their 2node flag cleared in recognition of the new member. Additional info: I manually changed expected votes to 3 on nodes 1 & 2, and the service was able to failover to node-3 (and back) with no problem. If node1 is rebooted, quorum is maintained and the services continue fine. After node-1 comes back into the cluster, its 2node flag was cleared and it showed the proper nodes/votes/quorum.
Created attachment 154966 [details] 2 & 3-node cluster.conf files, /var/log/messages on node-1 and node-3
Created attachment 154984 [details] Patch to fix Here's a patch to fix, it's pretty small. I'll check it into head for now.
Checked in to RHEL5 branch Checking in cmanccs.c; /cvs/cluster/cluster/cman/daemon/cmanccs.c,v <-- cmanccs.c new revision: 1.21.2.3; previous revision: 1.21.2.2 done Checking in commands.c; /cvs/cluster/cluster/cman/daemon/commands.c,v <-- commands.c new revision: 1.55.2.7; previous revision: 1.55.2.6 done
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux release. Since this bugzilla is in a component that is not approved for the current release, it has been closed with resolution deferred. You may reopen this bugzilla for consideration in the next release.
Errr, so what's happening here ? Are you saying the fix doesn't work or that it has not been packaged ? /confused.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2007-0575.html