Bug 460909 - Two new nodes with dirty flag don't see each other
Two new nodes with dirty flag don't see each other
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: cman (Show other bugs)
All Linux
urgent Severity medium
: rc
: ---
Assigned To: Christine Caulfield
Cluster QE
: ZStream
Depends On:
Blocks: 471273
  Show dependency treegraph
Reported: 2008-09-02 11:34 EDT by Christine Caulfield
Modified: 2010-10-23 00:14 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-01-20 16:50:07 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Small program to set the dirty flag in cman (117 bytes, text/x-csrc)
2008-09-02 11:36 EDT, Christine Caulfield
no flags Details
Proposed patch (1.74 KB, patch)
2008-09-02 11:37 EDT, Christine Caulfield
no flags Details | Diff

  None (edit)
Description Christine Caulfield 2008-09-02 11:34:57 EDT
Description of problem:

If two nodes start up in a partitioned network - ie they can't see each other, often because the switch separating them hasn't sorted out its multicast routes yet - and have the dirty flags set, when the network connection is restored the two nodes don't see each other. No node is fenced or even marked "disallowed".

Version-Release number of selected component (if applicable):

How reproducible:
Every time

Steps to Reproduce:
1. Separate the two nodes eg:
   on 131:  iptables -A INPUT -s -p udp -j DROP
   on 132:  iptables -A INPUT -s -p udp -j DROP
2. Join them to the cluster.
3. Set the dirty flag, I think fenced does this.
4. Join the cluster nodes with iptables -D INPUT 1

Actual results:

cman_tool nodes on both systems shows only the local node, and not the other. syslog shows that openais/Clm can see both nodes.

Expected results:

The other node shows up as "disallowed" or there is a fence race to kill one node.

Additional info:

This bug is similar to bz#443358

See also bz#460190
Comment 1 Christine Caulfield 2008-09-02 11:36:17 EDT
Created attachment 315566 [details]
Small program to set the dirty flag in cman
Comment 2 Christine Caulfield 2008-09-02 11:37:24 EDT
Created attachment 315567 [details]
Proposed patch

This patch should fix the problem. I'd like to see it tested rather a lot before releasing it though.
Comment 3 RHEL Product and Program Management 2008-09-09 11:46:52 EDT
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
Comment 4 Christine Caulfield 2008-09-10 04:07:46 EDT

commit 74721309f73dc6dc38abd07dc7c08e0ecb8ec602
Author: Christine Caulfield <ccaulfie@redhat.com>
Date:   Wed Sep 10 09:06:25 2008 +0100

cman: honour the dirty flag on a node we haven't seen before

Comment 10 errata-xmlrpc 2009-01-20 16:50:07 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.