Created attachment 1119189 [details]
pcs cluster report output

The syntax and documentation for colocation sets is quite confusing (even to me). I will work on the documentation, but to clarify for this case, this is my understanding:

* require-all=false is meaningful only for ordering sets, not colocation sets, so it is irrelevant here

* a colocation set with sequential=false, counterintuitively, does not add any colocation dependency between the items in that set. Instead, it allows the set to be in a colocation constraint with other sets, without its members being affected by each other. Therefore, it only makes sense if there is another set in the constraint.


A constraint like your first one, of the form:

  colocation set A B C sequential=false

is a null statement, with no effect.


A constraint like your second one, of the form:

  colocation set A B C sequential=false set D

means that A, B, and C must be colocated with D, but A, B, and C do not have any explicit relationship with each other (so any of them can fail without affecting the others).


Also, there is always some confusion between ordering constraints and colocation constraints. Colocation of "A with B" (including the colocation set equivalent) means that B will be assigned to a node first, but that does not mean that B will necessarily be started first. Without a separate ordering constraint, A and B may be started simultaneously (or even in reverse order) -- it just will be guaranteed to be on the same node.

I'll clone this BZ for pcs so we can have it reject require-all=false for colocation sets, and reject colocation sets with sequential=false unless there is another set in the constraint.

Thanks Ken, that make sense. Something like you write here should go to the documentation to avoid future confusion.

To elaborate further, if the goal is to place one resource with another if possible, but allow it to run even if the other resource isn't running, then a colocation constraint or set with a score less than INFINITY is sufficient. For example:

  pcs constraint colocation add B with A 1000

would place B with A if A is running, but allow B to run anywhere if A is not running. And:

  pcs constraint colocation set A B C setoptions score=1000

would place A first, then place B with A if possible (otherwise anywhere), then place C with B if possible (otherwise anywhere).

Reassigning to RHEL 7.2 docs. Same concerns apply to RHEL 6.

Upstream documentation has been updated: http://clusterlabs.org/doc/en-US/Pacemaker/1.0/html-single/Pacemaker_Explained/index.html#ch-constraints

The relevant upstream changes are: https://github.com/ClusterLabs/pacemaker/pull/927/files

The key items for RH docs:

* In 6.2.3. Ordered Resource Sets, role is not an option for these sets.

* In 6.3.3 Colocating Sets of Resources, action and require-all are not options for these sets.

* In both those sections, we should make it clearer that sequential=false only makes sense if multiple sets are listed in the constraint, otherwise the constraint has no effect.

* The description of require-all in ordered sets could be more detailed; upstream documentation should be a good starting point.

*** Bug 1311749 has been marked as a duplicate of this bug. ***

Ken:

I have made a few updates to the RHEL 7.3 version of the Pacemaker doc but this might not be quite enough explanation, I'm not sure.

The latest version of the doc is here:

http://jenkinscat.gsslab.pnq.redhat.com:8080/job/doc-Red_Hat_Enterprise_Linux-7-High_Availability_Add-On_Reference%20%28html-single%29/lastSuccessfulBuild/artifact/tmp/en-US/html-single/index.html#s2-resourceorderlist-HAAR

Ordered Resource Sets is Section 7.2.3

Colocating Sets of Resources is Section 7.3.3

Steven

7.2.3 Ordered Resource Sets:

It is probably worth mentioning that if the goal is to make a set of resources both colocated together and started in order, a resource group (section 6.5) is usually the best choice. Ordered sets allow the resources to be ordered, without necessarily being colocated on the same node.

I'm not sure if it needs to be changed, but where you say "following the options parameter", there is no options parameter explicitly specified, rather the options are simply listed after the set members. For example "pcs constraint order set A B sequential=false" (not "... options sequential=false"). (By contrast, "setoptions" is explicitly needed.)

The description for "sequential" for ordered sets should be along the lines of:

"sequential, which can be set to true or false to indicate whether the members of the set must be ordered relative to each other.

Setting sequential to false allows a set to be ordered relative to other sets in the ordering constraint, without its members being ordered relative to each other. Therefore, this option makes sense only if multiple sets are listed in the constraint; otherwise, the constraint has no effect."

The key distinction is that an ordered set is always ordered relative to other sets in the constraint, but sequential controls whether the members of the set are ordered relative to each other.


7.3.3 Colocating Sets of Resources:

Probably a similar reference to groups is useful here. Colocation sets allow the resources to be placed on the same node, without requiring them to be started in order.

The same comment for ordering applies about the "options parameter".

The description of "sequential" for colocated sets should be along the lines of:

"sequential, which can be set to true or false to indicate whether the members of the set must be colocated with each other.

Setting sequential to false allows the members of this set to be colocated with another set listed later in the constraint, regardless of which members of this set are active. Therefore, this option makes sense only if another set is listed after this one in the constraint; otherwise, the constraint has no effect."

One point of possible confusion that may be worth pointing out is that when listing members of a set, each member is colocated with the one *before* it. For example, "set A B" means "B is colocated with A". However, when listing multiple sets, each set is colocated with the one *after* it. For example, "set C D sequential=false set A B" means "set C D (where C and D have no relation between each other) is colocated with set A B (where B is colocated with A)".

Ken:

I have made the updates in comment 10.  I wasn't sure if your point of possible confusion was information you wanted me to rephrase or use verbatim but it seemed clear just as is so I left it as is.

If you want to check the updates they are here:

http://jenkinscat.gsslab.pnq.redhat.com:8080/job/doc-Red_Hat_Enterprise_Linux-7-High_Availability_Add-On_Reference%20%28html-single%29/lastSuccessfulBuild/artifact/tmp/en-US/html-single/index.html#s2-resourceorderlist-HAAR

No further updates from Ken and 7.3 is going out the door so while I'm pretty sure this is finished I'm moving it to 7.4 for now just to be sure Ken is ok with what we have before closing this.