Bug 1267265

Summary: A change in "crm_resource --set-parameter is-managed" introduces regression for Clone and M/S resources
Product: Red Hat Enterprise Linux 7 Reporter: Radek Steiger <rsteiger>
Component: pacemakerAssignee: Andrew Beekhof <abeekhof>
Status: CLOSED ERRATA QA Contact: cluster-qe <cluster-qe>
Severity: high Docs Contact:
Priority: high    
Version: 7.2CC: cfeist, cluster-maint, fdinitto, tlavigne, tojeline
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pacemaker-1.1.13-10.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1268801 (view as bug list) Environment:
Last Closed: 2015-11-19 12:12:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1268801    

Description Radek Steiger 2015-09-29 13:43:07 UTC 
> Description of problem:

In previous versions of pacemaker (1.1.12-22.el7) calling 'crm_resource --resource <primitive> --set-parameter is-managed' caused the attribute to be always assigned to the primitive element in CIB. After upgrading to recent pacemaker (1.1.13-8.el7) the crm_resource tool modifies the attribute only in the parent attribute if the primitive resource is cloned or M/S.

This introduces a regression when upgrading pacemaker from the older version to a newer one -- if the resource has been un-managed before the upgrade, it is not possible to set it managed again via crm_resource.


[root@virt-142 ~]# rpm -q pacemaker
pacemaker-1.1.12-22.el7.x86_64

[root@virt-142 ~]# pcs resource create stateful0 stateful --master
Warning: changing a monitor operation interval from 10 to 11 to make the operation unique
[root@virt-142 ~]# crm_resource --resource stateful0 --set-parameter is-managed --meta --parameter-value false

Produces:

      <master id="stateful0-master">
        <primitive class="ocf" id="stateful0" provider="pacemaker" type="Stateful">
          <instance_attributes id="stateful0-instance_attributes"/>
          <operations>
            ...
          </operations>
          <meta_attributes id="stateful0-meta_attributes">
            <nvpair id="stateful0-meta_attributes-is-managed" name="is-managed" value="false"/>
          </meta_attributes>
        </primitive>
      </master>

Upgrade the cluster and try to manage the resource:

[root@virt-142 ~]# pcs cluster stop --all && yum -y update && pcs cluster start --all
[root@virt-142 ~]# rpm -q pacemaker
pacemaker-1.1.13-8.el7.x86_64

root@virt-142 ~]# crm_resource --resource stateful0 --set-parameter is-managed --meta --parameter-value true
Updating 'stateful0-master' for 'stateful0'...
Set 'stateful0-master' option: id=stateful0-master-meta_attributes-is-managed set=stateful0-master-meta_attributes name=is-managed=true

Which will not set the resource back to be managed:

[root@virt-142 ~]# crm_mon -1 -D
Online: [ virt-142 virt-143 virt-144 virt-145 ]
 Master/Slave Set: stateful0-master [stateful0]
     stateful0	(ocf::pacemaker:Stateful):	Slave virt-144 (unmanaged)
     stateful0	(ocf::pacemaker:Stateful):	Slave virt-145 (unmanaged)
     stateful0	(ocf::pacemaker:Stateful):	Master virt-142 (unmanaged)
     stateful0	(ocf::pacemaker:Stateful):	Slave virt-143 (unmanaged)


The CIB looks like this:

      <master id="stateful0-master">
        <primitive class="ocf" id="stateful0" provider="pacemaker" type="Stateful">
          <instance_attributes id="stateful0-instance_attributes"/>
          <operations>
            ...
          </operations>
          <meta_attributes id="stateful0-meta_attributes">
            <nvpair id="stateful0-meta_attributes-is-managed" name="is-managed" value="false"/>
          </meta_attributes>
        </primitive>
        <meta_attributes id="stateful0-master-meta_attributes">
          <nvpair id="stateful0-master-meta_attributes-is-managed" name="is-managed" value="true"/>
        </meta_attributes>
      </master>


The attribute is now present twice and crm_resource doesn't care about the inner one if parent exists for the primitive. This also brings a problem to manage/unmanage commands in PCS due to a change of functionality that's not compatible backwards.



> Version-Release number of selected component (if applicable):

pacemaker-1.1.13-8.el7



> How reproducible:

Always


> Steps to Reproduce:

See description above.


> Actual results:

The new crm_resource only adds the attribute to parent element even if the primitive is referenced. I don't see any way to set/unset it to the primitive for cloned and M/S resources.


> Expected results:

Either there's needs to be a way to set/unset it for the primitive, or a change in the parent element needs to clear out the primitive attributes as well.
Comment 1 Andrew Beekhof 2015-09-30 07:37:07 UTC 
Expected usage: Users should always refer to the outer-most container when calling crm_resource.

Except users don't take the time to learn about that :-(
So we now allow them to refer to the uncloned resource, but it is a relatively new affordance and we try to help people set meta attributes in the correct place.
Which is what introduced the bug.

So I have fixed the bug reported here:

[05:11 PM] beekhof@fedora ~/Development/sources/pacemaker/devel ☺ # cp managed.xml foo.xml; CIB_file=./foo.xml tools/crm_resource --resource stateful0 --set-parameter is-managed --meta --parameter-value true ; cat foo.xml | grep nvpair
Set 'stateful0' option: id=stateful0-meta_attributes-is-managed name=is-managed=true
            <nvpair id="stateful0-meta_attributes-is-managed" name="is-managed" value="true"/>

New attributes still go to the parent by default:

[05:12 PM] beekhof@fedora ~/Development/sources/pacemaker/devel ☺ # cp managed.xml foo.xml; CIB_file=./foo.xml tools/crm_resource --resource stateful0 --set-parameter target-role --meta --parameter-value Stopped ; cat foo.xml | grep nvpair
Updating 'target-role' on 'stateful0-master' the parent of 'stateful0'
Set 'stateful0-master' option: id=stateful0-master-meta_attributes-target-role set=stateful0-master-meta_attributes name=target-role=Stopped
            <nvpair id="stateful0-meta_attributes-is-managed" name="is-managed" value="false"/>
          <nvpair id="stateful0-master-meta_attributes-target-role" name="target-role" value="Stopped"/>

However they can use --force if they really insist.

[05:13 PM] beekhof@fedora ~/Development/sources/pacemaker/devel ☺ # cp managed.xml foo.xml; CIB_file=./foo.xml tools/crm_resource --resource stateful0 --set-parameter target-role --meta --parameter-value Stopped --force ; cat foo.xml | grep nvpair
Forcing update of 'stateful0' instead of 'stateful0-master'
Set 'stateful0' option: id=stateful0-meta_attributes-target-role set=stateful0-meta_attributes name=target-role=Stopped
            <nvpair id="stateful0-meta_attributes-is-managed" name="is-managed" value="false"/>
            <nvpair id="stateful0-meta_attributes-target-role" name="target-role" value="Stopped"/>

And if they use the clone like they should have all along, they still get the right result:

[05:27 PM] beekhof@fedora ~/Development/sources/pacemaker/devel ☺ # cp managed.xml foo.xml; CIB_file=./foo.xml tools/crm_resource --resource stateful0-master --set-parameter is-managed --meta --parameter-value true ; cat foo.xml | grep nvpair
A value for 'is-managed' already exists in child 'stateful0', updating that instead of 'stateful0-master'
Set 'stateful0' option: id=stateful0-meta_attributes-is-managed name=is-managed=true
            <nvpair id="stateful0-meta_attributes-is-managed" name="is-managed" value="true"/>

Upstream patch is https://github.com/beekhof/pacemaker/commit/f367348
Comment 5 Andrew Beekhof 2015-10-08 02:35:44 UTC 
O-kay, here we go...

tl;dr - We now look for and summarise duplicates, plus the --force option now works for deletions and in the presence of duplicates.

Anything else?


# cp dups.xml foo.xml; CIB_file=./foo.xml tools/crm_resource -r stateful0-master --meta -p is-managed -v true
Multiple attributes match name=is-managed
  Value: false 	(id=stateful0-meta_attributes-is-managed)
  Value: false 	(id=stateful0-clone-meta_attributes-is-managed)
A value for 'is-managed' already exists in child 'stateful0', performing update on that instead of 'stateful0-master'
Set 'stateful0' option: id=stateful0-meta_attributes-is-managed name=is-managed=true

# cp dups.xml foo.xml; CIB_file=./foo.xml tools/crm_resource -r stateful0-master --meta -p is-managed -v true --force
Set 'stateful0-master' option: id=stateful0-clone-meta_attributes-is-managed name=is-managed=true

# cp dups.xml foo.xml; CIB_file=./foo.xml tools/crm_resource -r stateful0 --meta -p is-managed -v true
Multiple attributes match name=is-managed
  Value: false 	(id=stateful0-meta_attributes-is-managed)
  Value: false 	(id=stateful0-clone-meta_attributes-is-managed)
Set 'stateful0' option: id=stateful0-meta_attributes-is-managed name=is-managed=true

# cp dups.xml foo.xml; CIB_file=./foo.xml tools/crm_resource -r stateful0 --meta -p is-managed -v true --force
Set 'stateful0' option: id=stateful0-meta_attributes-is-managed name=is-managed=true


# cp managed.xml foo.xml; CIB_file=foo.xml tools/crm_resource -r stateful0-master --meta -p is-managed -v true
A value for 'is-managed' already exists in child 'stateful0', performing update on that instead of 'stateful0-master'
Set 'stateful0' option: id=stateful0-meta_attributes-is-managed name=is-managed=true

# cp managed.xml foo.xml; CIB_file=foo.xml tools/crm_resource -r stateful0-master --meta -p is-managed -v true --force
Set 'stateful0-master' option: id=stateful0-master-meta_attributes-is-managed set=stateful0-master-meta_attributes name=is-managed=true

# cp managed.xml foo.xml; CIB_file=foo.xml tools/crm_resource -r stateful0 --meta -p is-managed -v true
Set 'stateful0' option: id=stateful0-meta_attributes-is-managed name=is-managed=true

# cp managed.xml foo.xml; CIB_file=foo.xml tools/crm_resource -r stateful0 --meta -p is-managed -v true --force
Set 'stateful0' option: id=stateful0-meta_attributes-is-managed name=is-managed=true


# cp ../dups.xml foo.xml; CIB_file=./foo.xml tools/crm_resource -r stateful0 --meta -d is-managed
Multiple attributes match name=is-managed
  Value: false 	(id=stateful0-meta_attributes-is-managed)
  Value: false 	(id=stateful0-clone-meta_attributes-is-managed)

Deleted 'stateful0' option: id=stateful0-meta_attributes-is-managed name=is-managed

# cp ../dups.xml foo.xml; CIB_file=./foo.xml tools/crm_resource -r stateful0 --meta -d is-managed --force
Deleted 'stateful0' option: id=stateful0-meta_attributes-is-managed name=is-managed

# cp ../dups.xml foo.xml; CIB_file=./foo.xml tools/crm_resource -r stateful0-master --meta -d is-managed 
Multiple attributes match name=is-managed
  Value: false 	(id=stateful0-meta_attributes-is-managed)
  Value: false 	(id=stateful0-clone-meta_attributes-is-managed)

A value for 'is-managed' already exists in child 'stateful0', performing delete on that instead of 'stateful0-master'
Deleted 'stateful0' option: id=stateful0-meta_attributes-is-managed name=is-managed

# cp ../dups.xml foo.xml; CIB_file=./foo.xml tools/crm_resource -r stateful0-master --meta -d is-managed --force
Deleted 'stateful0-master' option: id=stateful0-clone-meta_attributes-is-managed name=is-managed

As a bonus if there is no duplicate but you specify the "wrong" resource:

# CIB_file=./foo.xml tools/crm_resource -r stateful0-master --meta -d is-managed 
A value for 'is-managed' already exists in child 'stateful0', performing delete on that instead of 'stateful0-master'
Deleted 'stateful0' option: id=stateful0-meta_attributes-is-managed name=is-managed

and

# CIB_file=./foo.xml tools/crm_resource -r stateful0 --meta -d is-managed 
Performing delete of 'is-managed' on 'stateful0-master', the parent of 'stateful0'
Deleted 'stateful0-master' option: id=stateful0-clone-meta_attributes-is-managed name=is-managed
Comment 6 Andrew Beekhof 2015-10-14 23:13:43 UTC 
Forgot to update bug status
Comment 7 Andrew Beekhof 2015-10-15 00:50:18 UTC 
Adding to the errata
Comment 11 errata-xmlrpc 2015-11-19 12:12:01 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-2383.html