Bug 217408 - cman still shows deleted cluster nodes
cman still shows deleted cluster nodes
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: cman (Show other bugs)
5.0
All Linux
medium Severity medium
: ---
: ---
Assigned To: Christine Caulfield
Cluster QE
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2006-11-27 15:01 EST by Kiersten (Kerri) Anderson
Modified: 2009-04-16 18:29 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-12-06 16:28:04 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Kiersten (Kerri) Anderson 2006-11-27 15:01:10 EST
Description of problem:
After removing a node reference from the cluster.conf and updating the file
through ccs_tool, cman_tool nodes command still shows a reference to the nodes.
 This differs from ccs_tool's view of the cluster.

[root@kanderso-xen-01 cluster]# ccs_tool lsnode

Cluster name: ka-xen-cluster, config_version: 29

Nodename                        Votes Nodeid Fencetype
kanderso-xen-01.lab.msp.redhat.com   1    1    xvm
kanderso-xen-02.lab.msp.redhat.com   1    2    xvm
kanderso-xen-03.lab.msp.redhat.com   1    3    xvm
kanderso-xen-04.lab.msp.redhat.com   1    4    xvm
kanderso-xen-05.lab.msp.redhat.com   1    5    xvm
kanderso-xen-06.lab.msp.redhat.com   1   10    xvm
kanderso-xen-07.lab.msp.redhat.com   1   11    xvm
kanderso-xen-08.lab.msp.redhat.com   1   12    xvm
kanderso-xen-09.lab.msp.redhat.com   1   13    xvm
kanderso-xen-22.lab.msp.redhat.com   1    6    xvm
kanderso-xen-23.lab.msp.redhat.com   1    7    xvm
kanderso-xen-24.lab.msp.redhat.com   1    8    xvm
kanderso-xen-25.lab.msp.redhat.com   1    9    xvm
kanderso-xen-26.lab.msp.redhat.com   1   14    xvm
kanderso-xen-27.lab.msp.redhat.com   1   15    xvm
kanderso-xen-28.lab.msp.redhat.com   1   16    xvm
[root@kanderso-xen-01 cluster]# cman_tool nodes
Node  Sts   Inc   Joined               Name
   1   M      4   2006-11-22 16:38:18  kanderso-xen-01.lab.msp.redhat.com
   2   M      8   2006-11-22 16:38:20  kanderso-xen-02.lab.msp.redhat.com
   3   M     16   2006-11-22 16:38:22  kanderso-xen-03.lab.msp.redhat.com
   4   M     12   2006-11-22 16:38:21  kanderso-xen-04.lab.msp.redhat.com
   5   M     24   2006-11-22 16:38:24  kanderso-xen-05.lab.msp.redhat.com
   6   M     20   2006-11-22 16:38:23  kanderso-xen-22.lab.msp.redhat.com
   7   M     28   2006-11-22 16:38:25  kanderso-xen-23.lab.msp.redhat.com
   8   M     28   2006-11-22 16:38:25  kanderso-xen-24.lab.msp.redhat.com
   9   M     32   2006-11-22 16:38:26  kanderso-xen-25.lab.msp.redhat.com
  10   M     84   2006-11-27 11:54:37  kanderso-xen-06.lab.msp.redhat.com
  11   M     40   2006-11-27 11:04:19  kanderso-xen-07.lab.msp.redhat.com
  12   M     44   2006-11-27 11:05:51  kanderso-xen-08.lab.msp.redhat.com
  13   M     48   2006-11-27 11:06:22  kanderso-xen-09.lab.msp.redhat.com
  14   M     52   2006-11-27 11:06:36  kanderso-xen-26.lab.msp.redhat.com
  15   M     56   2006-11-27 11:06:56  kanderso-xen-27.lab.msp.redhat.com
  16   M     60   2006-11-27 11:07:17  kanderso-xen-28.lab.msp.redhat.com
  17   X     64                        kanderso-xen-29.lab.msp.redhat.com


At this point, am unable to update the cluster.conf file in the cluster to add
the node back in since that node is no longer running the cluster software. 
ccs_tool update cluster.conf.new fails, not sure if this might be a seperate defect.

cman_tool status shows the correct number of nodes and quorum count:

[root@kanderso-xen-01 cluster]# cman_tool status
Version: 6.0.1
Config Version: 28
Cluster Name: ka-xen-cluster
Cluster Id: 15028
Cluster Member: Yes
Cluster Generation: 88
Membership state: Cluster-Member
Nodes: 16
Expected votes: 9
Total votes: 16
Quorum: 9  
Active subsystems: 8
Flags: 
Ports Bound: 0 11 177  
Node name: kanderso-xen-01.lab.msp.redhat.com
Node ID: 1
Multicast addresses: 239.192.58.238 
Node addresses: 10.15.85.21 

All nodes remaining in the cluster also have the following groupd status:
[root@kanderso-xen-01 cluster]# group_tool -v
type             level name       id       state node id local_done
fence            0     default    00010003 FAIL_START_WAIT 17 1100100003 0
[1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16]
dlm              1     clvmd      00010009 FAIL_START_WAIT 17 1100100003 0
[1 2 3 4 5 6 7 8 9]
dlm              1     rgmanager  00020003 FAIL_START_WAIT 17 1100100003 0
[1 2 3 4 5 6 7 8 9]


Version-Release number of selected component (if applicable):


How reproducible:
Haven't tried reproducing yet, but should be pretty simple

Steps to Reproduce:
1. ccs_tool delete node_name
2. fence_node nodename
3. cman_tool nodes
  
Actual results:


Expected results:


Additional info:
Comment 1 Christine Caulfield 2006-11-28 04:27:05 EST
This is true. cman NEVER deletes nodes. It never did.

What cman_tool shows you is not the contents of ccs, it's the list of nodes that
it knows about. You can add to this from CCS but you can't remove nodes.

The main reason the old cman did this was to avoid people configuring themselves
(and possiblty the cluster) into confusion by attempting to reuse node names or
node IDs. Now that we insist on providing static node IDs for cluster nodes the
need for this is less, but doig  afull delta between CCS and the internal
cluster node list might stretch the time spent in the AIS confchg call so it
would probably need to go in a separate thread. 

(roughly translated that means its not trivial!)

Comment 2 Kiersten (Kerri) Anderson 2006-11-28 10:59:17 EST
I need to go back and retry this one with the proper use of the cman_tool
version command and see if any of the problems still remain.  My bet is that it
will be okay at that point.  Setting to needinfo to revalidate that a problem
really exists.
Comment 3 Rob Kenna 2006-11-28 22:10:12 EST
Ack.  Looks like this will be works-as-spec'ed anyway
Comment 4 Kiersten (Kerri) Anderson 2006-12-06 16:28:04 EST
Closing as works as speced.  I did have to run an additional command -
cman_tool expected -e 16

to set the expected votes correctly in the cluster after doing the delete node
and fencing the node.
Comment 5 Nate Straz 2007-12-13 12:22:14 EST
Moving all RHCS ver 5 bugs to RHEL 5 so we can remove RHCS v5 which never existed.

Note You need to log in before you can comment on or make changes to this bug.