Bug 460909 - Two new nodes with dirty flag don't see each other
Two new nodes with dirty flag don't see each other
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: cman (Show other bugs)
5.2
All Linux
urgent Severity medium
: rc
: ---
Assigned To: Christine Caulfield
Cluster QE
: ZStream
Depends On:
Blocks: 471273
  Show dependency treegraph
 
Reported: 2008-09-02 11:34 EDT by Christine Caulfield
Modified: 2010-10-23 00:14 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-01-20 16:50:07 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Small program to set the dirty flag in cman (117 bytes, text/x-csrc)
2008-09-02 11:36 EDT, Christine Caulfield
no flags Details
Proposed patch (1.74 KB, patch)
2008-09-02 11:37 EDT, Christine Caulfield
no flags Details | Diff

  None (edit)
Description Christine Caulfield 2008-09-02 11:34:57 EDT
Description of problem:

If two nodes start up in a partitioned network - ie they can't see each other, often because the switch separating them hasn't sorted out its multicast routes yet - and have the dirty flags set, when the network connection is restored the two nodes don't see each other. No node is fenced or even marked "disallowed".


Version-Release number of selected component (if applicable):
5.0+

How reproducible:
Every time

Steps to Reproduce:
1. Separate the two nodes eg:
   on 131:  iptables -A INPUT -s 10.15.84.132 -p udp -j DROP
   on 132:  iptables -A INPUT -s 10.15.84.131 -p udp -j DROP
2. Join them to the cluster.
3. Set the dirty flag, I think fenced does this.
4. Join the cluster nodes with iptables -D INPUT 1

Actual results:

cman_tool nodes on both systems shows only the local node, and not the other. syslog shows that openais/Clm can see both nodes.


Expected results:

The other node shows up as "disallowed" or there is a fence race to kill one node.

Additional info:

This bug is similar to bz#443358

See also bz#460190
Comment 1 Christine Caulfield 2008-09-02 11:36:17 EDT
Created attachment 315566 [details]
Small program to set the dirty flag in cman
Comment 2 Christine Caulfield 2008-09-02 11:37:24 EDT
Created attachment 315567 [details]
Proposed patch

This patch should fix the problem. I'd like to see it tested rather a lot before releasing it though.
Comment 3 RHEL Product and Program Management 2008-09-09 11:46:52 EDT
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 4 Christine Caulfield 2008-09-10 04:07:46 EDT
On RHEL5:

commit 74721309f73dc6dc38abd07dc7c08e0ecb8ec602
Author: Christine Caulfield <ccaulfie@redhat.com>
Date:   Wed Sep 10 09:06:25 2008 +0100

cman: honour the dirty flag on a node we haven't seen before

and STABLE2
Comment 10 errata-xmlrpc 2009-01-20 16:50:07 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-0189.html

Note You need to log in before you can comment on or make changes to this bug.