Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 896660

Summary: Group resources with constraints are not handled properly by pcs resource delete
Product: Red Hat Enterprise Linux 6 Reporter: Jaroslav Kortus <jkortus>
Component: pcsAssignee: Chris Feist <cfeist>
Status: CLOSED ERRATA QA Contact: Cluster QE <mspqa-list>
Severity: high Docs Contact:
Priority: high    
Version: 6.3CC: av82.msstate, cfeist, cluster-maint, dvossel, mnovacek, rsteiger, slevine
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pcs-0.9.75-1.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-11-21 11:49:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 960054    
Attachments:
Description Flags
example cib
none
cluster cib after 'pcs resource move haa-group-0' has been issued none

Description Jaroslav Kortus 2013-01-17 17:10:30 UTC 
Description of problem:
pcs group resources (group or Master/Slave sets) cannot be removed by pcs resource delete.

Unlike "plain" resources, pcs does not remove the related constraints first. The returned error code also does not correspond to error condition.

$ pcs resource delete webserver
Deleting Resource (and group) - webserver
ERROR: Unable to remove resource 'webserver' (do constraints exist?)
$ echo $?
0

This behaviour is a regression to previous versions, where it did work with group resource.

Version-Release number of selected component (if applicable):
pcs-0.9.26-9.el6.noarch

How reproducible:
100%

Steps to Reproduce:
1. group couple of resources to a group
2. issue pcs move <added resource> (or create any other constraint)
3. pcs resource delete
4. command #3 will fail on last resource in the group
  
Actual results:
* last resource of group resource not removed if there is a constraint
* error code still 0

Expected results:
* constraints automatically removed as with plain non-group resources
* error code should be > 0 if error occurs

Additional info:
Comment 2 David Vossel 2013-01-17 17:31:29 UTC 
Created attachment 680405 [details]
example cib

Using the attached example cib, here are some test cases that seem like they should work but don't.

----------------
# pcs -f 896660_cib.xml resource delete Q
Deleting Resource - Q
Unable to remove resource: Q, it may still be referenced in constraints.

# pcs -f 896660_cib.xml resource delete W
Removing Constraint - colocation-W-master-X-master-INFINITY
Call failed: Protocol not supported

Deleting Resource - W
Unable to remove resource: W, it may still be referenced in constraints.

# pcs -f 896660_cib.xml resource delete W-master
ERROR: Unable to update cib
Call failed: Protocol not supported

----------

Also, I'm curious about the below command.  The only way to delete a group is to delete all the resources within it.  That makes sense to me, although perhaps a syntax shortcut to delete the entire group with a --force option might be useful at some point.

It would be nice however to have error output that explains that GROUP1 is in fact a resource, but that GROUP1 will be removed when all the member resources are deleted.

# pcs -f backup resource delete GROUP1
Error: Resource does not exist.

might be better with this output...

# pcs -f backup resource delete GROUP1
Error: GROUP1 resource group is automatically deleted after all rsc members are deleted or removed.
Comment 3 David Vossel 2013-01-17 22:58:29 UTC 
Sorry, ignore comment 2.

I was using the latest version of pcs, but accidentally had an outdated version of pacemaker installed.  Weird things were happening as a result.

Using that same cib xml I uploaded I see this.

--- Problem removing master resource when colocation rule exists, no warning as to what the problem is.

# pcs -f 896660_cib.xml resource delete W-master
ERROR: Unable to update cib

but... deleting the primitive the Master rsc is based off of works fine.

#pcs -f tmp resource delete W
Removing Constraint - colocation-W-master-X-master-INFINITY
Deleting Resource - W

--- Same problem reported in the description, where last member of a group can't be removed if a constraint exists for the group.  This isn't so bad though since the error message points to a work around.

#pcs -f 896660_cib.xml resource delete Q
Deleting Resource - Q
# pcs -f 896660_cib.xml resource delete R
Deleting Resource - R
# pcs -f 896660_cib.xml resource delete S
Deleting Resource (and group) - S
ERROR: Unable to remove resource 'S' (do constraints exist?)


-- Vossel
Comment 4 RHEL Program Management 2013-01-21 06:47:29 UTC 
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.
Comment 5 Chris Feist 2013-02-21 00:19:33 UTC 
These issues should be fixed upstream (with the latest commit):

https://github.com/feist/pcs/commit/c9d540a9b6ce47ea05a2da0447d54894c3159996
Comment 10 Chris Feist 2013-08-13 21:20:59 UTC 
[root@ask-02 ~]# pcs -f test.xml resource create Test Dummy
[root@ask-02 ~]# pcs -f test.xml resource group add TestGroup Test
[root@ask-02 ~]# pcs -f test.xml resource
 Resource Group: TestGroup
     Test	(ocf::heartbeat:Dummy):	Stopped 
[root@ask-02 ~]# pcs -f test.xml resource delete Test
Deleting Resource (and group) - Test
[root@ask-02 ~]# pcs -f test.xml resource
NO resources configured
Comment 12 michal novacek 2013-08-20 13:26:02 UTC 
Created attachment 788501 [details]
cluster cib after 'pcs resource move haa-group-0' has been issued
Comment 13 michal novacek 2013-08-20 13:26:47 UTC 
The bug seem to remain, test in comment #10 is not done according to initial
reproducer (no move issued) and therefore there are no constraint on a group.
Removing resources from the group with no constraints attached works and have
worked before.

commands used to check that bug has been corrected:

$ pcs status
...
Full list of resources:

 virt-fencing   (stonith:fence_xvm):    Started virt-062 
 Resource Group: haa-group-0
     ip-0       (ocf::heartbeat:IPaddr2):       Started virt-061 
     lvm-0      (ocf::heartbeat:LVM):   Started virt-061 
     fs-0       (ocf::heartbeat:Filesystem):    Started virt-061 
     apache-0   (ocf::heartbeat:apache):        Started virt-061 
...

$ pcs resource move haa-group-0 
$ pcs cluster cib > /tmp/cluster.cib.moved
# pcs -f /tmp/cluster.cib.moved status
...
Full list of resources:

 virt-fencing   (stonith:fence_xvm):    Started virt-062.cluster-qe.lab.eng.brq.redhat.com 
 Resource Group: haa-group-0
     ip-0       (ocf::heartbeat:IPaddr2):       Started virt-063.cluster-qe.lab.eng.brq.redhat.com 
     lvm-0      (ocf::heartbeat:LVM):   Started virt-063.cluster-qe.lab.eng.brq.redhat.com 
     fs-0       (ocf::heartbeat:Filesystem):    Started virt-063.cluster-qe.lab.eng.brq.redhat.com 
     apache-0   (ocf::heartbeat:apache):        Started virt-063.cluster-qe.lab.eng.brq.redhat.com 
...

# for a in apache-0 fs-0 lvm-0 ip-0; do \
pcs -f /tmp/cluster.cib.moved resource delete $a; \
done
Deleting Resource - apache-0
Deleting Resource - fs-0
Deleting Resource - lvm-0
Deleting Resource (and group) - ip-0
Error: Unable to remove resource 'ip-0' (do constraints exist?)
Comment 14 av82.msstate 2013-08-20 17:03:44 UTC 
Constraint on group fails , pcs-0.9.26.10.e16_4.1.noarch
Comment 15 Chris Feist 2013-08-20 22:30:50 UTC 
Before Fix:
[root@ask-02 ~]# rpm -q pcs
pcs-0.9.70-1.el6.noarch
[root@ask-02 ~]# pcs resource create D1 Dummy --group DGroup
[root@ask-02 ~]# pcs resource create D2 Dummy --group DGroup
[root@ask-02 ~]# pcs resource create D3 Dummy --group DGroup
[root@ask-02 ~]# pcs resource create D4 Dummy --group DGroup
[root@ask-02 ~]# pcs status
Cluster name: 
Last updated: Tue Aug 20 17:46:15 2013
Last change: Tue Aug 20 17:45:48 2013 via cibadmin on ask-02
Stack: cman
Current DC: ask-02 - partition with quorum
Version: 1.1.10-6.el6-368c726
2 Nodes configured
4 Resources configured


Online: [ ask-02 ask-03 ]

Full list of resources:

 Resource Group: DGroup
     D1	(ocf::heartbeat:Dummy):	Started ask-02 
     D2	(ocf::heartbeat:Dummy):	Started ask-02 
     D3	(ocf::heartbeat:Dummy):	Started ask-02 
     D4	(ocf::heartbeat:Dummy):	Started ask-02 


PCSD Status:
Error: no nodes found in corosync.conf
[root@ask-02 ~]# pcs resource move DGroup
[root@ask-02 ~]# pcs constraint
Location Constraints:
  Resource: DGroup
    Disabled on: ask-02 
Ordering Constraints:
Colocation Constraints:
[root@ask-02 ~]# pcs resource delete D1
Deleting Resource - D1
[root@ask-02 ~]# pcs resource delete D2
Deleting Resource - D2
[root@ask-02 ~]# pcs resource delete D3
Deleting Resource - D3
[root@ask-02 ~]# pcs resource delete D4
Deleting Resource (and group) - D4
Error: Unable to remove resource 'D4' (do constraints exist?)


After Fix:
[root@ask-02 ~]# rpm -q pcs
pcs-0.9.73-1.el6.noarch
[root@ask-02 ~]# pcs resource create D1 Dummy --group DGroup
[root@ask-02 ~]# pcs resource create D2 Dummy --group DGroup
[root@ask-02 ~]# pcs resource create D3 Dummy --group DGroup
[root@ask-02 ~]# pcs resource create D4 Dummy --group DGroup
[root@ask-02 ~]# pcs status
Cluster name: testcs
Last updated: Tue Aug 20 17:48:14 2013
Last change: Tue Aug 20 17:48:12 2013 via cibadmin on ask-02
Stack: cman
Current DC: ask-02 - partition with quorum
Version: 1.1.10-6.el6-368c726
2 Nodes configured
4 Resources configured


Online: [ ask-02 ask-03 ]

Full list of resources:

 Resource Group: DGroup
     D1	(ocf::heartbeat:Dummy):	Started ask-02 
     D2	(ocf::heartbeat:Dummy):	Started ask-02 
     D3	(ocf::heartbeat:Dummy):	Started ask-02 
     D4	(ocf::heartbeat:Dummy):	Started ask-02 


[root@ask-02 ~]# pcs resource move DGroup
[root@ask-02 ~]# pcs constraint
Location Constraints:
  Resource: DGroup
    Disabled on: ask-02 
Ordering Constraints:
Colocation Constraints:
[root@ask-02 ~]# pcs resource delete D1
Deleting Resource - D1
[root@ask-02 ~]# pcs resource delete D2
Deleting Resource - D2
[root@ask-02 ~]# pcs resource delete D3
Deleting Resource - D3
[root@ask-02 ~]# pcs resource delete D4
Removing Constraint - cli-ban-DGroup-on-ask-02
Deleting Resource (and group) - D4
[root@ask-02 ~]# pcs resource
NO resources configured
Comment 19 Chris Feist 2013-08-27 22:35:00 UTC 
Before Fix:
[root@ask-02 test]# rpm -q pcs
pcs-0.9.73-1.el6.noarch
[root@ask-02 test]# pcs resource create dummy0 Dummy
[root@ask-02 test]# pcs resource create dummy1 Dummy
[root@ask-02 test]# pcs resource group add group0 dummy0 dummy1
[root@ask-02 test]# pcs constraint order set dummy0 dummy1
[root@ask-02 test]# pcs resource delete dummy1
Deleting Resource - dummy1
Error: unable to remove resource: dummy1, it may still be referenced in constraints.
[root@ask-02 test]# pcs constraint remove pcs_rsc_order
[root@ask-02 test]# pcs resource group remove group0
[root@ask-02 test]# pcs resource delete dummy0
Deleting Resource - dummy0
[root@ask-02 test]# pcs resource delete dummy1
Deleting Resource - dummy1

... Next Test ...

[root@ask-02 test]# pcs resource create dummy0 Dummy
[root@ask-02 test]# pcs resource create dummy1 Dummy
[root@ask-02 test]# pcs resource group add group0 dummy0 dummy1
[root@ask-02 test]# pcs resource move group0
[root@ask-02 test]# pcs resource master group0
[root@ask-02 test]# pcs resource delete dummy0
Deleting Resource - dummy0
[root@ask-02 test]# pcs resource delete dummy1
Deleting Resource (and group and M/S) - dummy1
Error: Unable to remove resource 'dummy1' (do constraints exist?)
[root@ask-02 test]# pcs constraint remove cli-ban-group0-on-ask-02
[root@ask-02 test]# pcs resource delete dummy1


After Fix:
[root@ask-02 test]# rpm -q pcs
pcs-0.9.75-1.el6.noarch
[root@ask-02 test]# pcs resource create dummy0 Dummy
[root@ask-02 test]# pcs resource create dummy1 Dummy
[root@ask-02 test]# pcs resource group add group0 dummy0 dummy1
[root@ask-02 test]# pcs constraint order set dummy0 dummy1
[root@ask-02 test]# pcs resource delete dummy1
Removing dummy1 from set pcs_rsc_set
Deleting Resource - dummy1
[root@ask-02 test]# pcs resource delete dummy0
Removing dummy0 from set pcs_rsc_set
Removing set pcs_rsc_set
Removing constraint pcs_rsc_order
Deleting Resource (and group) - dummy0
[root@ask-02 test]# pcs status
Cluster name: testcs
Last updated: Tue Aug 27 16:18:16 2013
Last change: Tue Aug 27 16:18:12 2013 via cibadmin on ask-02
Stack: cman
Current DC: ask-02 - partition with quorum
Version: 1.1.10-6.el6-368c726
2 Nodes configured
0 Resources configured


Online: [ ask-02 ask-03 ]

Full list of resources:



[root@ask-02 test]# pcs resource create dummy0 Dummy
[root@ask-02 test]# pcs resource create dummy1 Dummy
[root@ask-02 test]# pcs resource group add group0 dummy0 dummy1
[root@ask-02 test]# pcs resource move group0
[root@ask-02 test]# pcs resource master group0
[root@ask-02 test]# pcs resource delete dummy0
Deleting Resource - dummy0
[root@ask-02 test]# pcs resource delete dummy1
Deleting Resource (and group and M/S) - dummy1
Comment 22 errata-xmlrpc 2013-11-21 11:49:41 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1633.html