RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1578820 - Fix moving and banning bundles with one instance
Summary: Fix moving and banning bundles with one instance
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: pacemaker
Version: 8.3
Hardware: Unspecified
OS: Unspecified
low
medium
Target Milestone: rc
: 8.9
Assignee: Ken Gaillot
QA Contact: cluster-qe
Steven J. Levine
URL:
Whiteboard:
Depends On:
Blocks: 1520665 1578789 1621899 2233771
TreeView+ depends on / blocked
 
Reported: 2018-05-16 12:30 UTC by Tomas Jelinek
Modified: 2023-11-14 16:50 UTC (History)
5 users (show)

Fixed In Version: pacemaker-2.1.6-1.el8
Doc Type: Bug Fix
Doc Text:
.The `crm_resource` command now allows banning or moving a bundle with only a single active replica Previously, when the `crm_resource` command checked where a bundle with a single replica was active, the command counted both the node where the container was active and the guest node that was created for the container itself. As a result, the `crm_resource` command would not ban or move a bundle with a single active replica. With this fix, the `crm_resource` command now only counts nodes where a bundle's containers are active when determining the number of active replicas.
Clone Of:
Environment:
Last Closed: 2023-11-14 15:32:34 UTC
Type: Bug
Target Upstream Version: 2.1.6
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker CLUSTERQE-6690 0 None None None 2023-05-16 13:11:21 UTC
Red Hat Knowledge Base (Solution) 6972370 0 None None None 2023-06-06 14:21:27 UTC
Red Hat Product Errata RHEA-2023:6970 0 None None None 2023-11-14 15:33:21 UTC

Description Tomas Jelinek 2018-05-16 12:30:39 UTC 
Description of problem:
Pacemaker should allow moving and banning bundle resources with one instance (replicas=1) in the same way it allows moving and banning clone resources with one instance (clone-max=1).


Version-Release number of selected component (if applicable):
pacemaker-1.1.16-12.el7.x86_64


How reproducible:
always


Steps to Reproduce:

moving without specifying a target node
---------------------------------------

DummyBundle is a bundle with replicas=1:

[root@virt-143 ~]# crm_resource --resource DummyBundle --move
Resource 'DummyBundle' not moved: active in 2 locations.
You can prevent 'DummyBundle' from running on a specific location with: --ban --node <name>
Error performing operation: Invalid argument

> crm_resource gives us misinformation about the number of locations being active. There should only be one active due to replicas=1.


dummy1-clone is a clone with clone-max=1:

[root@virt-143 ~]# crm_resource --resource dummy1-clone --move
WARNING: Creating rsc_location constraint 'cli-ban-dummy1-clone-on-virt-143' with a score of -INFINITY for resource dummy1-clone on virt-143.
	This will prevent dummy1-clone from running on virt-143 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin
	This will be the case even if virt-143 is the last node in the cluster
	This message can be disabled with --quiet

> When targeting clone ID, a clone with single instance passed and WAS moved with crm_resource.


dummy1-clone is a clone with clone-max=2:

[root@virt-143 ~]# crm_resource --resource dummy1-clone --move
Resource 'dummy1-clone' not moved: active in 2 locations.
You can prevent 'dummy1-clone' from running on a specific location with: --ban --node <name>
Error performing operation: Invalid argument


moving with specifying a target node
------------------------------------

DummyBundle is a bundle with replicas=1:

[root@virt-143 ~]# crm_resource --resource DummyBundle --move --node virt-145
Resource 'DummyBundle' not moved: active on multiple nodes
Error performing operation: Invalid argument

> Just like in the previous case the crm_resource gives us a misinformation about the number of locations being active. There should only be one active due to replicas=1.


dummy1-clone is a clone with clone-max=1:

[root@virt-143 ~]# crm_resource --resource dummy1-clone --move --node virt-146
[root@virt-143 ~]# pcs constraint --full
Location Constraints:
  Resource: dummy1-clone
    Enabled on: virt-146 (score:INFINITY) (role: Started) (id:cli-prefer-dummy1-clone)

> When targeting clone ID, a clone with single instance passed and WAS moved with crm_resource.


dummy1-clone is a clone with clone-max=2:

[root@virt-143 ~]# crm_resource --resource dummy1-clone --move --node virt-146
[root@virt-143 ~]# pcs constraint --full
Location Constraints:
  Resource: dummy1-clone
    Enabled on: virt-146 (score:INFINITY) (role: Started) (id:cli-prefer-dummy1-clone)


banning
-------

DummyBundle is a bundle with replicas=1:

[root@virt-143 ~]# crm_resource --resource DummyBundle --ban
Resource 'DummyBundle' not moved: active in 2 locations.
You can prevent 'DummyBundle' from running on a specific location with: --ban --node <name>
Error performing operation: Invalid argument

> Just like in the previous case the crm_resource gives us a misinformation about the number of locations being active. There should only be one active due to replicas=1.


dummy1-clone is a clone with clone-max=1:

[root@virt-143 ~]# crm_resource --resource dummy1-clone --ban
WARNING: Creating rsc_location constraint 'cli-ban-dummy1-clone-on-virt-144' with a score of -INFINITY for resource dummy1-clone on virt-144.
	This will prevent dummy1-clone from running on virt-144 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin
	This will be the case even if virt-144 is the last node in the cluster
	This message can be disabled with --quiet
[root@virt-143 ~]# pcs constraint --full
Location Constraints:
  Resource: dummy1-clone
    Disabled on: virt-144 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-clone-on-virt-144)

> When targeting clone ID, a clone with single instance passed and WAS moved with crm_resource.


dummy1-clone is a clone with clone-max=2:

[root@virt-143 ~]# crm_resource --resource dummy1-clone --ban
Resource 'dummy1-clone' not moved: active in 2 locations.
You can prevent 'dummy1-clone' from running on a specific location with: --ban --node <name>
Error performing operation: Invalid argument
Comment 1 Ken Gaillot 2020-05-06 00:26:53 UTC 
Moving to RHEL 8 only
Comment 5 RHEL Program Management 2021-01-08 07:29:26 UTC 
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.
Comment 8 Ken Gaillot 2023-01-23 17:25:43 UTC 
FYI, the problem here is that Pacemaker considers the overall bundle instance to be active on 2 (or potentially even 3) nodes because its implicit resources are active on different nodes. For example the instance's container could be on node1, its remote connection could be on node2, and its containerized primitive is considered to be running on the guest node created by starting the container.

It should be reasonably straightforward to special-case bundles when counting active instances, to count only containers, not remote connections and containerized primitives. Location constraints for a bundle apply only to its containers, so it makes sense to count only them for this purpose.
Comment 9 Ken Gaillot 2023-05-02 21:49:25 UTC 
Fixed in upstream main branch as of commit 905dda99
Comment 13 Markéta Smazová 2023-06-20 12:12:55 UTC 
after fix
--------

>   [root@virt-502 ~]# rpm -q pacemaker
>   pacemaker-2.1.6-1.el8.x86_64

Setup cluster and a bundle resource with one instance (replicas=1):
>   [root@virt-502 ~]# pcs status
>   Cluster name: STSRHTS287
>   Cluster Summary:
>     * Stack: corosync (Pacemaker is running)
>     * Current DC: virt-502 (version 2.1.6-1.el8-6fdc9deea29) - partition with quorum
>     * Last updated: Mon Jun 19 16:02:02 2023 on virt-502
>     * Last change:  Mon Jun 19 16:01:58 2023 by root via cibadmin on virt-502
>     * 2 nodes configured
>     * 4 resource instances configured

>   Node List:
>     * Online: [ virt-502 virt-503 ]

>   Full List of Resources:
>     * fence-virt-502	(stonith:fence_xvm):	 Started virt-502
>     * fence-virt-503	(stonith:fence_xvm):	 Started virt-503
>     * Container bundle: TestBundle1 [redis:test]:
>       * TestBundle1-podman-0 (127.0.0.2)	(ocf::heartbeat:podman):	 Started virt-502

>   Daemon Status:
>     corosync: active/enabled
>     pacemaker: active/enabled
>     pcsd: active/enabled

>   [root@virt-502 ~]# pcs resource config
>   Bundle: TestBundle1
>     Podman: image=redis:test replicas=1
>     Network: ip-range-start=127.0.0.2 host-interface=lo host-netmask=8
>     Port Mapping:
>       port=80 (httpd-port)

Move bundle resource without specifying a target node:
>   [root@virt-502 ~]# crm_resource --resource TestBundle1 --move
>   WARNING: Creating rsc_location constraint 'cli-ban-TestBundle1-on-virt-502' with a score of -INFINITY for resource TestBundle1 on virt-502.
>       This will prevent TestBundle1 from running on virt-502 until the constraint is removed using the clear option or by editing the CIB with an appropriate tool
>       This will be the case even if virt-502 is the last node in the cluster

>   [root@virt-502 ~]# pcs resource
>     * Container bundle: TestBundle1 [redis:test]:
>       * TestBundle1-podman-0 (127.0.0.2)	(ocf::heartbeat:podman):	 Started virt-503

>   [root@virt-502 ~]# pcs constraint
>   Location Constraints:
>     Resource: TestBundle1
>       Disabled on:
>         Node: virt-502 (score:-INFINITY) (role:Started)
>   Ordering Constraints:
>   Colocation Constraints:
>   Ticket Constraints:

RESULT: OK, bundle was moved to another node.


Move bundle resource and specify a target node:
>   [root@virt-502 ~]# crm_resource --resource TestBundle1 --move --node virt-502

>   [root@virt-502 ~]# pcs resource
>     * Container bundle: TestBundle1 [redis:test]:
>       * TestBundle1-podman-0 (127.0.0.2)	(ocf::heartbeat:podman):	 Starting virt-502

>   [root@virt-502 ~]# pcs constraint
>   Location Constraints:
>     Resource: TestBundle1
>       Enabled on:
>         Node: virt-502 (score:INFINITY) (role:Started)
>   Ordering Constraints:
>   Colocation Constraints:
>   Ticket Constraints:

RESULT: OK, bundle was moved to specified node.


Ban bundle resource:
>   [root@virt-502 ~]# crm_resource --resource TestBundle1 --ban
>   WARNING: Creating rsc_location constraint 'cli-ban-TestBundle1-on-virt-502' with a score of -INFINITY for resource TestBundle1 on virt-502.
>       This will prevent TestBundle1 from running on virt-502 until the constraint is removed using the clear option or by editing the CIB with an appropriate tool
>       This will be the case even if virt-502 is the last node in the cluster

>   [root@virt-502 ~]# pcs resource
>     * Container bundle: TestBundle1 [redis:test]:
>       * TestBundle1-podman-0 (127.0.0.2)	(ocf::heartbeat:podman):	 Started virt-503

>   [root@virt-502 ~]# pcs constraint
>   Location Constraints:
>     Resource: TestBundle1
>       Enabled on:
>         Node: virt-502 (score:INFINITY) (role:Started)
>       Disabled on:
>         Node: virt-502 (score:-INFINITY) (role:Started)
>   Ordering Constraints:
>   Colocation Constraints:
>   Ticket Constraints:

RESULT: OK, bundle was banned from one of the nodes.

marking verified in pacemaker-2.1.6-1.el8.x86_64
Comment 24 errata-xmlrpc 2023-11-14 15:32:34 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (pacemaker bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:6970
Note You need to log in before you can comment on or make changes to this bug.