Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
For bugs related to Red Hat Enterprise Linux 5 product line. The current stable release is 5.10. For Red Hat Enterprise Linux 6 and above, please visit Red Hat JIRA https://issues.redhat.com/secure/CreateIssue!default.jspa?pid=12332745 to report new issues.

Bug 453000

Summary: Rgmanager stops heartbeat interface is Virtual IP is substring match to the broadcast address
Product: Red Hat Enterprise Linux 5 Reporter: Shane Bradley <sbradley>
Component: rgmanagerAssignee: Lon Hohberger <lhh>
Status: CLOSED ERRATA QA Contact: Cluster QE <mspqa-list>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.2.zCC: cluster-maint, edamato, gavinf, keith.fralick, Stuart.Kirk, tao
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-01-20 20:56:32 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
rgmanager mismatch ip none

Description Shane Bradley 2008-06-26 14:48:35 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008061712 Fedora/3.0-1.fc9 Firefox/3.0

Description of problem:

Problem description: On a cluster node where there is an rgmanager IP resource that is a substring match to the broadcast address of any interface, stopping or relocating that service will bring down that entire interface.  If this happens to be the interface used for cluster communication then the node misses its heartbeats and gets fenced.

Example: ashprdgfs01 has eth0 = 172.20.200.21/24.  This means the broadcast address is 172.20.200.255.  Stopping the following resource:

  <ip address="172.20.200.25" monitor_link="1"/>

causes the wrong ip address to be removed:

  Jun 23 15:18:27 ashprdgfs01 clurgmgrd[3889]: <notice> Stopping service service:VIP
  Jun 23 15:18:27 ashprdgfs01 clurgmgrd: [3889]: <info> Removing IPv4 address 172.20.200.21/24 from eth0

Since this is the main ip for eth0, this kills cluster communication and the node gets fenced:

  Jun 23 15:18:42 ashprdgfs02 kernel: dlm: closing connection to node 2
  Jun 23 15:18:42 ashprdgfs02 fenced[3199]: fencing node "ashprdgfs01.gspt.net"

The problem is on line 714 of ip.sh:

  addr=`/sbin/ip addr list | grep "$addr" | head -n 1 | awk '{print $2}'`

Because 172.20.200.25 is a substring of 172.20.200.255, this actually returns 2 lines:

   inet 172.20.200.21/24 brd 172.20.200.255 scope global eth0
   inet 172.20.200.25/24 scope global secondary eth0

and the first one is chosen mistakenly, causing it to be the one removed (718):

   /sbin/ip -f inet addr del dev $dev $addr

The address we want will always be follwed by a '/' for the subnet, so this can be fixed easily like so:

   addr=`/sbin/ip addr list | grep "$addr/" | head -n 1 | awk '{print $2}'`

Version-Release number of selected component (if applicable):
rgmanager-2.0.38-2

How reproducible:
Always


Steps to Reproduce:
1) Configure an interface with a 255.255.255.0 netmask
2) Configure an IP resource on that interface with last octet of .25 or .2
3) Stop the service:

     # clusvcadm -s vip

Actual Results:
VIP service should stop, leaving the main ip on that interface intact. If this is the heartbeat interface then the node loses communication with cluster and node is fenced off.



Expected Results:
The entire interface is taken down.

Additional info:
Workaround is to not use VIP that is substring of broadcast.

Comment 1 Shane Bradley 2008-06-26 15:02:20 UTC
Created attachment 310344 [details]
rgmanager mismatch ip

Comment 2 Lon Hohberger 2008-06-26 17:58:06 UTC
Applied.

On RHEL4, that line isn't even present.

Comment 3 RHEL Program Management 2008-07-03 18:21:38 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 4 Fedora Update System 2008-07-30 20:03:55 UTC
gfs2-utils-2.03.05-1.fc9, rgmanager-2.03.05-1.fc9, cman-2.03.05-1.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 5 Lon Hohberger 2008-08-01 16:13:36 UTC
This breaks when netmasks are used.


Comment 6 Lon Hohberger 2008-08-01 16:16:29 UTC
@@ -702,7 +702,7 @@
 			if [ "${addr/\/*/}" != "$ifaddr" ]; then
 			        continue
 			fi
-			addr=`/sbin/ip addr list | grep "$addr" | head -n 1 | awk '{print $2}'`
+			addr=`/sbin/ip addr list | grep "$addr/" | head -n 1 | awk '{print $2}'`

... should be:

+			addr=`/sbin/ip addr list | grep "$ifaddr/" | head -n 1 | awk '{print $2}'`
                                                         ^^^^^^^^



Comment 7 Lon Hohberger 2008-08-22 18:04:24 UTC
http://git.fedorahosted.org/git/?p=cluster.git;a=commit;h=40e34ea406411c10a53aa1a147a0c94a62c3d12b

Ancillary patch has been pushed to RHEL5 branch.

Comment 8 Lon Hohberger 2008-08-22 18:07:06 UTC
Pushed to STABLE2 and master branches as well.

Comment 9 Lon Hohberger 2008-09-03 15:31:15 UTC
*** Bug 460541 has been marked as a duplicate of this bug. ***

Comment 12 errata-xmlrpc 2009-01-20 20:56:32 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-0101.html