Upstream:
https://github.com/ClusterLabs/pacemaker/pull/1270/commits/f2948e786548942b20c9724b59d243b6178e5411

But I wonder if we need instance_attributes as well:
https://github.com/ClusterLabs/pacemaker/pull/1270/commits/f2948e786548942b20c9724b59d243b6178e5411#r114756431

I do think we should implement these as meta-attributes, so tooling doesn't need to have special cases for bundles. My idea is that meta-attributes set on a bundle will have no effect on the bundle per se, but will be inherited by all its components. That way, target-role, is-managed, priority, resource-stickiness, requires, etc, can all work as expected (hopefully with a minimum of extra coding). This inheritance is comparable to what is done for groups.

Instance attributes will be allowed by the schema to gain the benefits of reduced code duplication in the schema, but they will be undocumented, and ignored by the source code.

I hope no special special-casing would be needed as they are
optional, afterall :-)

Unfortunately, there's a precedence with superfluous instance_attributes
-- groups.

It might make sense then to procede with tolerating them, and queue the
cleanup for both for version 3 of schemas.

* precedent, proceed

To clarify, the choice is between implementing is-managed, target-role, etc., as meta-attributes (as is done with every other type of pacemaker resource), or as XML attributes of the bundle tag. If we choose the former, all existing tooling will work without modification. If we choose the latter, all existing tooling will need to be updated with special code paths for bundles.

The argument for using XML attributes is that it would have been a better design from the start, so that meta-properties allowed by pacemaker would be enumerated in (and enforceable by) the schema. However, I think it would be too much of a burden for tooling to switch that model now. It would be something to consider for "pacemaker 2.0" if we ever get to that point.

Groups intentionally have instance_attributes that are inheritable by the group members, though that's only useful for a group of identical resources. That doesn't apply to bundles since the components are always different.

Understood, there was a confusion arising from intermingling two things.

I can only talk of instance_attributes, and I think it would be good to
proceed with dropping instance_attributes for groups and bundles in
3.0 schema at latest -- the corner case with groups can still be
addressed by unrolling, e.g. using templates for simplicity, in XSLT,
while for bundles case, it would plain drop it, as it hasn't ever been
used.  Notice that no change in code would be needed, beside perhaps
spotting and eliminating new dead code.

Upstream pacemaker now supports setting meta-attributes on bundles, and having them inherited by the components. At least for target-role, this works as expected.

Initial tests with is-managed did not work as expected, so I'm continuing to investigate that. But at least some support will already be available with the current proposed 7.4 packages.

Implemented upstream with https://github.com/ClusterLabs/pacemaker/pull/1283

QA: In case the pcs interface for setting meta-attributes on bundles is not ready, here is how to test it without pcs:

1. Configure a cluster with a bundle resource (for example, the test procedure for Bug 1432722).

2. Ensure you have your EDITOR shell variable set and exported (e.g. "export EDITOR=vi").

3. Run "pcs cluster edit" to manually edit the CIB.
3a. Find your bundle resource, and enclosed in the bundle tag, add this:

        <meta_attributes id="test-meta">
          <nvpair id="test-target-role" name="target-role" value="Started"/>
          <nvpair id="test-is-managed" name="is-managed" value="true"/>
        </meta_attributes>
3b. Save and exit.

4. As desired, run "pcs cluster edit" and try combinations of target-role = Started or Stopped and is-managed = true or false, watching "pcs status" to see how it affects the cluster. The most important thing is that all the component parts of the bundle respect the target-role (i.e. all of them start or stop as specified).

Additionally, Michele Baldessari and myself are testing the new features from this build, so I can say that's it's working as expected for us.

We're following different instructions [1] to deploy a cluster with containerized ocf resources, and this is the result:

[root@rhelz ~]# crm_mon -1
Stack: corosync
Current DC: rhelz (version 1.1.16-11.el7-94ff4df) - partition with quorum
Last updated: Wed Jun 21 09:48:16 2017
Last change: Wed Jun 21 09:29:42 2017 by root via cibadmin on rhelz

4 nodes configured
16 resources configured

Online: [ rhelz ]
GuestOnline: [ galera-bundle-0@rhelz rabbitmq-bundle-0@rhelz redis-bundle-0@rhelz ]

Active resources:

 Docker container: rabbitmq-bundle [192.168.24.1:8787/rhosp12/openstack-rabbitmq-docker:2017-06-19.1]
   rabbitmq-bundle-0    (ocf::heartbeat:rabbitmq-cluster):      Started rhelz
 Docker container: galera-bundle [192.168.24.1:8787/rhosp12/openstack-mariadb-docker:2017-06-19.1]
   galera-bundle-0      (ocf::heartbeat:galera):        Master rhelz
 Docker container: redis-bundle [192.168.24.1:8787/rhosp12/openstack-redis-docker:2017-06-19.1]
   redis-bundle-0       (ocf::heartbeat:redis): Master rhelz
 ip-192.168.122.254     (ocf::heartbeat:IPaddr2):       Started rhelz
 ip-192.168.122.250     (ocf::heartbeat:IPaddr2):       Started rhelz
 ip-192.168.122.249     (ocf::heartbeat:IPaddr2):       Started rhelz
 ip-192.168.122.253     (ocf::heartbeat:IPaddr2):       Started rhelz
 ip-192.168.122.247     (ocf::heartbeat:IPaddr2):       Started rhelz
 ip-192.168.122.248     (ocf::heartbeat:IPaddr2):       Started rhelz
 Docker container: haproxy-bundle [192.168.24.1:8787/rhosp12/openstack-haproxy-docker:2017-06-19.1]
   haproxy-bundle-docker-0      (ocf::heartbeat:docker):        Started rhelz


In this setup, the bundle resources are created in "disabled" mode, we then create location constraint and associate them with the bundle. Ultimately we enable the bundles.

All those operation are implemented via the regular pcs argument "--disable" and "--enable", implemented for bundles in https://bugzilla.redhat.com/show_bug.cgi?id=1447910.
Internally, those pcs operation make use of the target-role meta attribute, so this confirms that this new package fixes the bug.

[1] https://github.com/dciabrin/undercloud_ha_containers

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1862