Bug 361371 - RHEL5U1beta: reboot service cman on one node can cause inconsistent cluster status.
Summary: RHEL5U1beta: reboot service cman on one node can cause inconsistent cluster s...
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: cman
Version: 5.0
Hardware: All
OS: Linux
high
high
Target Milestone: ---
: ---
Assignee: Christine Caulfield
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2007-11-01 08:41 UTC by Zhenyong(Jerry) Jiang
Modified: 2013-07-29 00:44 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2008-07-07 14:54:00 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
message from gfs1 (182.20 KB, text/plain)
2007-11-05 09:15 UTC, Zhenyong(Jerry) Jiang
no flags Details
message from gfs2 (171.54 KB, text/plain)
2007-11-05 09:16 UTC, Zhenyong(Jerry) Jiang
no flags Details

Description Zhenyong(Jerry) Jiang 2007-11-01 08:41:22 UTC
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:

Always

Steps to Reproduce:

1. setup a tow-nodes cluster on RHEL5U1beta,(or RHEL5GA) 

2. make sure the cluster status is ok on both nodes.

3. restart service cman on node-a ( /etc/init.d/cman restart )

4. check the cluster status on both nodes, the command of clustat will show
different status.
  
Actual results:

node-a(in my example, it called gfs1)

  Member Name                        ID   Status
  ------ ----                        ---- ------
  gfs1                                  1 Online
  gfs2                                  2 Online


node-b(in my example, it called gfs2)

  Member Name                        ID   Status
  ------ ----                        ---- ------
  gfs1                                  1 Offline
  gfs2                                  2 Online

Expected results:

I think the status should be same.

Additional info:

Comment 1 Christine Caulfield 2007-11-01 08:57:12 UTC
Can you post the output of 'cman_tool status' and 'cman_tool nodes' from bothe
systems please ?

Comment 2 Zhenyong(Jerry) Jiang 2007-11-05 09:12:02 UTC
I restart cman on gfs1.

[root@gfs1 ~]# clustat 
msg_open: No such file or directory
Member Status: Quorate

  Member Name                        ID   Status
  ------ ----                        ---- ------
  gfs1                                  1 Online, Local
  gfs2                                  2 Online

[root@gfs1 ~]# 
[root@gfs1 ~]# 
[root@gfs1 ~]# 
[root@gfs1 ~]#  > /var/log/messages
[root@gfs1 ~]# /etc/init.d/cman restart
Stopping cluster: 
   Stopping fencing... done
   Stopping cman... done
   Stopping ccsd... done
   Unmounting configfs... done
                                                           [  OK  ]
Starting cluster: 
   Loading modules... done
   Mounting configfs... done
   Starting ccsd... done
   Starting cman... done
   Starting daemons... done
   Starting fencing... done
                                                           [  OK  ]
[root@gfs1 ~]# clustat
msg_open: No such file or directory
Member Status: Quorate

  Member Name                        ID   Status
  ------ ----                        ---- ------
  gfs1                                  1 Online, Local
  gfs2                                  2 Online

[root@gfs1 ~]# 



And this is shown on gfs2.

[root@gfs2 ~]# clustat 
msg_open: No such file or directory
Member Status: Quorate

  Member Name                        ID   Status
  ------ ----                        ---- ------
  gfs1                                  1 Online
  gfs2                                  2 Online, Local

[root@gfs2 ~]# << At this time, cman was restarted on gfs1 >>
[root@gfs2 ~]# clustat 
msg_open: No such file or directory
Member Status: Quorate

  Member Name                        ID   Status
  ------ ----                        ---- ------
  gfs1                                  1 Online
  gfs2                                  2 Online, Local

Comment 3 Zhenyong(Jerry) Jiang 2007-11-05 09:15:52 UTC
Created attachment 247811 [details]
message from gfs1

message from gfs1

Comment 4 Zhenyong(Jerry) Jiang 2007-11-05 09:16:31 UTC
Created attachment 247821 [details]
message from gfs2

message from gfs2

Comment 5 Christine Caulfield 2007-11-05 09:27:06 UTC
I really need the cman_tool outputs. This was I can determine whether the
problem is in cman itself, or clustat/rgmanager.

The logs you have provided hint that openais is not at fault, but it's hard to
tell without the cman_tool information.


Note You need to log in before you can comment on or make changes to this bug.