Bug 736084

Summary: rgmanager reports successful start of a service tree when members have failed
Product: Red Hat Enterprise Linux 6 Reporter: Madison Kelly <mkelly>
Component: resource-agentsAssignee: Chris Feist <cfeist>
Status: CLOSED NOTABUG QA Contact: Cluster QE <mspqa-list>
Severity: high Docs Contact:
Priority: low    
Version: 6.0CC: agk, cluster-maint, lhh
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-01 21:40:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 756082    

Description Madison Kelly 2011-09-06 16:03:03 UTC
Description of problem:

I've got a resource tree of:

<script ref="drbd">
	<script ref="clvmd">
		<script ref="gfs2">
			<script ref="libvirtd"/>
		</script>
	</script>
</script>

When DRBD was misconfigured (moved the servers, didn't update the 'address' values), I started the services. One one node, DRBD stayed StandAlone/Primary, so it was able to start all of the services and was parted as "Started". However, on the second node which stayed "StandAlone/Secondary", none of the child services could start. That is, clvmd and gfs2 should have failed. Despite this, rgmanager still reported a "Started" state for it's version of the resource tree.

Version-Release number of selected component (if applicable):

resource-agents-3.0.12-15.el6_0.1.x86_64
rgmanager-3.0.12-10.el6.x86_64
cman-3.0.12-23.el6_0.6.x86_64

How reproducible:

100%

Steps to Reproduce:
1. Create a 2-node cluster, create two failoverdomains (each containing one of the nodes), then create two services, each being for one of the failover domains using the resource tree above.
2. Create a dual-primary drbd resource (like this: https://alteeve.com/w/Red_Hat_Cluster_Service_2_Tutorial#Configuring_.2Fetc.2Fdrbd.conf <- effectively the same on EL6/DRBD 8.3.11 except resource are in separate files). Once up, use the DRBD resource to back a clustered LVM, create an LVM and format it as GFS2.
3. Stop one node's service first, then stop the other node. Now change the 'address' entry in the drbd's resource and try to start the services on both nodes. It will succeed on the node you shut down last, but should fail on the node you stopped first. Note that both show Started, despite Clustered LVM not seeing it's PV and GFS2 failing to mount.
  
Actual results:

Reports "Started" on failure.

Expected results:

Report "Failed" on failure.

Additional info:

I know that DRBD is not supported under EL6, but this is a problem with the resource-agents and/or rgmanager. In this case, I discovered this issue using DRBD, but a failed service, like GFS2, should fail the whole resource tree.

Comment 3 Lon Hohberger 2011-12-01 21:40:30 UTC
I think the problem here is that clvmd and gfs2 init scripts don't return an error if there are "zero" CLVM VGs or "zero" gfs2 volumes to mount; this isn't a specific resource-agents issue.

For example, manually specifying the gfs2 file system (using clusterfs resource) would have caused a failure on mount.