Bug 1930578

Description of problem:

See the description below, but note that this applies to clones in general, not just to promotable clones.

Copied from comment 10 of BZ1925681:
~~~
I think we have additional problems with crm_resource's handling of clone instances (i.e., only using the first clone instance). The following test of the restart command is with the current master branch.

  * Clone Set: stateful-clone [stateful] (promotable):
    * Masters: [ node2 ]
    * Slaves: [ node1 ]

# crm_resource --restart --resource stateful --node node1
Performing update of 'target-role' on 'stateful-clone', the parent of 'stateful:0'
Set 'stateful-clone' option: id=stateful-clone-meta_attributes-target-role set=stateful-clone-meta_attributes name=target-role value=stopped
Waiting for 1 resources to stop:
 * stateful-clone
Performing delete of 'target-role' on 'stateful-clone', the parent of 'stateful:0'
Deleted 'stateful-clone' option: id=stateful-clone-meta_attributes-target-role name=target-role
Waiting for 1 resources to start again:
 * stateful-clone


# crm_resource --restart --resource stateful --node node2
stateful is not running on node2 and so cannot be restarted
crm_resource: Error performing operation: No such device or address

With trace enabled:
(resource_is_running_on)        trace: Resource stateful:0 is not running on: node2
stateful is not running on node2 and so cannot be restarted



cli_resource_restart has a similar issue to resource_search_list_default. It first checks whether rsc is running on the specified host. If not, it returns ENXIO. The problem is that the value of `rsc` is the one found earlier in main() by pe_find_resource_with_flags -- which, as we can see in comment 9, returns the first clone instance. Of course, this instance (stateful:0 in my example) will only be running on at most one node. So if stateful:0 is running on node 1 and stateful:1 is running on node 2, then an attempt to restart stateful on node 2 will fail.

    if(resource_is_running_on(rsc, host) == FALSE) {
        const char *id = rsc->clone_name?rsc->clone_name:rsc->id;
        if(host) {
            out->err(out, "%s is not running on %s and so cannot be restarted", id, host);
        } else {
            out->err(out, "%s is not running anywhere and so cannot be restarted", id);
        }
        return ENXIO;
    }
~~~

-----

Version-Release number of selected component (if applicable):

pacemaker-2.0.4-6.el8_3.1

-----

How reproducible:

Always

-----

Steps to Reproduce:
1. Create a clone resource (e.g., named "stateful") with no constraints so that it's allowed to run on all nodes.
2. Run `crm_resource --restart` for the resource on each cluster node as shown in the Description. (E.g., `crm_resource --restart -r stateful -N node1`).

-----

Actual results:

The resource is restarted on only one node. On the other nodes, the command fails with "crm_resource: Error performing operation: No such device or address".

-----

Expected results:

The resource is restarted on each node when the command is run for that node.