Bug 743218

Summary: Cluster problem: "hard" service dependencies
Product: Red Hat Enterprise Linux 6 Reporter: Frank Danapfel <fdanapfe>
Component: rgmanagerAssignee: Lon Hohberger <lhh>
Status: CLOSED ERRATA QA Contact: Cluster QE <mspqa-list>
Severity: high Docs Contact:
Priority: high    
Version: 6.1CC: cluster-maint, jentrena, jkortus, mjuricek
Target Milestone: rc   
Target Release: 6.2   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: rgmanager-3.0.12.1-6.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 743214 Environment:
Last Closed: 2012-06-20 14:12:50 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 743214    
Bug Blocks: 756082    
Attachments:
Description Flags
Logs none

Description Frank Danapfel 2011-10-04 09:35:57 UTC 
+++ This bug was initially created as a clone of Bug #743214 +++

Description of problem:
Two Cluster Services are defined. The first service "sapgt1ci" starts th
e database and the central SAP instance.
The second cluster service "sapgt1ap1" starts another SAP application instance.
 "sapgt1ap1" has a hard dependency on sapgt1ci.

While executing the cluster tests / failover tests we encountered a bug:

Setup: Cluster Server lxcn1gt1 running no services and cluster server lxcn2gt1 running both sapgt1ci and sapgt1ap1.

Cluster node lxcn2gt1 (the one running everything) gets switched off (power off). Now both services get migrated to the remaining node. The start sequence is correct and matches the configured dependency. But after both services were successfully started sapgt1ap1 gets stopped again (!) while

rgmanager prints a message about "lost dependency" into the syslog. After sapgt1ap1 was successfully stopped it gets started again. This time everything stays up.

The re-start of sapgt1ap1 should not occur.

With the services swapped in /etc/cluster/cluster.conf (now sapgt1ap1 is defined first and the sapgtci second) we see a slightly different behavior:

If both services run on the same cluster node (in our test case lxcn2gt1-hb) and this node gets powered off....:

The remaining node (lxcn1gt1-hb) first tries to start sapgt1ap1 (completely ignoring the hard dependency on sapgt1ci!!) - this start attempt fails without the central instance running.

Then it starts sapgt1ci (SAP DB and central instance) and then it starts sapgt1ap1 again (which works this time).

To recapture the initial problem with the service definition of sapgt1ci first in /etc/cluster/cluster.conf and sapgtap1 second:

During the failover of both services the remaining node

- first starts sapgt1ci
- then starts sapgt1ap1
- then stops sapgt1ap1 (message about missing dependency)
- then starts sapgt1ap1 again

Both start sequences are flawed.....

Nevertheless all instances were started in both cases (with an extra step in between).


Version-Release number of selected component (if applicable):
rgmanager-3.0.12-11.el6_1.1.x86_64              

How reproducible:
always
Comment 1 RHEL Program Management 2011-10-07 16:04:15 UTC 
Since RHEL 6.2 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.

Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.
Comment 5 Lon Hohberger 2012-02-22 14:48:40 UTC 
http://git.fedorahosted.org/git?p=cluster.git;a=commit;h=fb5d3eea42ea69cec1d37fd4d8018e32a36d2c37
http://git.fedorahosted.org/git?p=cluster.git;a=commit;h=32f3bbc58694b22ce2e672ed0b93c8c205d54f8d
http://git.fedorahosted.org/git?p=cluster.git;a=commit;h=2353fc5a8b4f590d97e10e0c6cd2c91506eb853a
http://git.fedorahosted.org/git?p=cluster.git;a=commit;h=4758c35d2a9862bd7a8ac79d5c2239b6a79ca0bf
http://git.fedorahosted.org/git?p=cluster.git;a=commit;h=4fbd15844e11e3677e865779e57b5943ca6f3c99
Comment 6 Lon Hohberger 2012-02-29 22:53:32 UTC 
I tested all 3 cases across two configurations identical to bug 743214 ; all passed with the same results (RHEL6 logging looks a bit different).
Comment 7 Lon Hohberger 2012-02-29 22:54:08 UTC 
Created attachment 566665 [details]
Logs
Comment 11 errata-xmlrpc 2012-06-20 14:12:50 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0897.html