Bug 1578789
| Summary: | Fix moving / banning clone and bundle resources | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Tomas Jelinek <tojeline> | |
| Component: | pcs | Assignee: | Tomas Jelinek <tojeline> | |
| Status: | CLOSED MIGRATED | QA Contact: | cluster-qe <cluster-qe> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | medium | |||
| Version: | 8.3 | CC: | cluster-maint, idevat, mlisik, mpospisi, nhostako, nwahl, omular, sbradley, tojeline | |
| Target Milestone: | rc | Keywords: | MigratedToJIRA, Triaged | |
| Target Release: | 8.10 | Flags: | pm-rhel:
mirror+
|
|
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | Bug Fix | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2233771 (view as bug list) | Environment: | ||
| Last Closed: | 2023-09-22 18:32:42 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1578820, 2233771 | |||
| Bug Blocks: | 1520665, 1621899 | |||
The bz1578820 we depend on got moved to RHEL 8 only. Summary after bz1578820 has been resolved: 'pcs resource ban' works as described in comment 0. Such behavior is fine and there is nothing to be fixed. Pcs delegates the banning to crm_resource, including detection of number of location a resource is active in. 'pcs resource move-with constraint' and 'pcs resource move' is needs to be fixed: * Moving / banning an inner bundle resource from its bundle should be disallowed in pcs. Currently, pcs doesn't check for this and creates a constraint. The constraint has no effect, as expected. * Moving a 1-replica bundle works, but pcs does not allow it. There is a check in a validator in 'pcs resource move' command which prevents moving all bundle resources. As moving a 1-replica bundle works in pacemaker, an moving multi-replica bundles is prevented by pacemaker, the check in pcs can be removed completely. * Moving a clone resource works the same as moving a bundle resource and is disallowed by pcs in the same way as well. Removing this limitation from pcs should be considered. Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug. This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there. Due to differences in account names between systems, some fields were not replicated. Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information. To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer. You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like: "Bugzilla Bug" = 1234567 In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information. |
Description of problem: * moving and banning an inner bundle resource * pcs should exit with an error saying moving / banning inner bundle resources is not allowed * moving / banning a pacemaker internal bundle resource * works OK, pcs says the resource doesn't exist (it's not in the CIB) * moving and banning a bundle resource * does not currently work in crm_resource even for 1-instance bundles: Resource 'DummyBundle' not moved: active in 2 locations. * moving and banning a clone resource * works in crm_resource for 1-instance clones for both a clone and its inner resource * for multi-instance clones, moving a resource in a clone creates a constraint, whereas moving a clone resource exits with an error * pcs should not allow moving / banning a resource in a clone, then it can simply run crm_resource every time and based on its output inform a user if a clone could or couldn't be moved / banned * another option is to automatically switch to moving / banning a bundle / clone if the user tries to move / ban a resource in the bundle / clone Version-Release number of selected component (if applicable): pcs-0.9.162-5.el7.x86_64 How reproducible: always, easily Steps to Reproduce: dummy0 is a resource inside DummyBundle bundle with replicas=1 -------------------------------------------------------------- [root@virt-143 ~]# pcs resource move dummy0 Warning: Creating location constraint cli-ban-dummy0-on-DummyBundle-0 with a score of -INFINITY for resource dummy0 on node DummyBundle-0. This will prevent dummy0 from running on DummyBundle-0 until the constraint is removed. This will be the case even if DummyBundle-0 is the last node in the cluster. [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy0 Disabled on: DummyBundle-0 (score:-INFINITY) (role: Started) (id:cli-ban-dummy0-on-DummyBundle-0) [root@virt-143 ~]# crm_resource --resource dummy0 --move WARNING: Creating rsc_location constraint 'cli-ban-dummy0-on-DummyBundle-0' with a score of -INFINITY for resource dummy0 on DummyBundle-0. This will prevent dummy0 from running on DummyBundle-0 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if DummyBundle-0 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy0 Disabled on: DummyBundle-0 (score:-INFINITY) (role: Started) (id:cli-ban-dummy0-on-DummyBundle-0) > Constraint has been created, but resource WAS NOT moved. The reason is obvious as DummyBundle-0 is not an actual node. This is a possible place for improvement. [root@virt-143 ~]# pcs resource move DummyBundle-0 Error: DummyBundle-0 is not a valid resource [root@virt-143 ~]# crm_resource --resource DummyBundle-0 --move WARNING: Creating rsc_location constraint 'cli-ban-DummyBundle-0-on-virt-143' with a score of -INFINITY for resource DummyBundle-0 on virt-143. This will prevent DummyBundle-0 from running on virt-143 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-143 is the last node in the cluster This message can be disabled with --quiet Error performing operation: Update does not conform to the configured schema > Neither pcs not crm_resource is able to target the DummyBundle-0 pseudo resource which is expected. [root@virt-143 ~]# pcs resource move DummyBundle Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource DummyBundle --move Resource 'DummyBundle' not moved: active in 2 locations. You can prevent 'DummyBundle' from running on a specific location with: --ban --node <name> Error performing operation: Invalid argument > In this situation pcs simply shouts for trying to move a clone, while crm_resource gives us something else - a misinformation about the number of locations being active. There should only be one active due to replicas=1. Let's see how well the pure clones will do: dummy1 is a clone with clone-max=1 ---------------------------------- [root@virt-143 ~]# pcs resource move dummy1 Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1 --move WARNING: Creating rsc_location constraint 'cli-ban-dummy1-on-virt-145' with a score of -INFINITY for resource dummy1 on virt-145. This will prevent dummy1 from running on virt-145 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-145 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-145 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-145) > A clone with single instance was refused to move by pcs, but actually passed and WAS moved with crm_resource. There might be a room for improvement in pcs. [root@virt-143 ~]# pcs resource move dummy1-clone Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1-clone --move WARNING: Creating rsc_location constraint 'cli-ban-dummy1-clone-on-virt-143' with a score of -INFINITY for resource dummy1-clone on virt-143. This will prevent dummy1-clone from running on virt-143 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-143 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1-clone Disabled on: virt-143 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-clone-on-virt-143) > When targeting clone ID, once again a clone with single instance was refused to move by pcs, but passed and WAS moved with crm_resource. dummy1 is a clone with clone-max=2 ---------------------------------- [root@virt-143 ~]# pcs resource move dummy1 Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1 --move WARNING: Creating rsc_location constraint 'cli-ban-dummy1-on-virt-143' with a score of -INFINITY for resource dummy1 on virt-143. This will prevent dummy1 from running on virt-143 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-143 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-143 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-143) [root@virt-143 ~]# crm_resource --resource dummy1 --move WARNING: Creating rsc_location constraint 'cli-ban-dummy1-on-virt-145' with a score of -INFINITY for resource dummy1 on virt-145. This will prevent dummy1 from running on virt-145 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-145 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-143 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-143) Disabled on: virt-145 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-145) > With more than one clone instance crm_resource does surprisingly perform a constraint-based move by moving ONE of the instance. Even if ran twice in a row. [root@virt-143 ~]# pcs resource move dummy1-clone Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1-clone --move Resource 'dummy1-clone' not moved: active in 2 locations. You can prevent 'dummy1-clone' from running on a specific location with: --ban --node <name> Error performing operation: Invalid argument > Here with clone ID and two instances crm_resource won't do us a favor anymore. Pcs keeps printing the same error as before. So, the conclusion for 'resource move': - pcs is not really up in sync with crm_resource behavior in terms of being able to move a single clone instance - support for moving a bundle in crm_resource is not on par with being able to move a clone, at least in case of just one instance being present due to: - 2 active locations being improperly reported - the move constraint of a primitive is incorrectly targeted to a non-existent DummyBundle-0 node. It is worth pointing out that creating a pure location constraint for DummyBundle will move the only existing bundle instance to the desired node as one would expect. I yet have to test the behavior of 'resource ban'. Performing tests of move with node specification. dummy0 is a resource inside DummyBundle bundle with replicas=1 -------------------------------------------------------------- [root@virt-143 ~]# pcs resource move dummy0 virt-145 [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy0 Enabled on: virt-145 (score:INFINITY) (role: Started) (id:cli-prefer-dummy0) [root@virt-143 ~]# crm_resource --resource dummy0 --move --node virt-145 [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy0 Enabled on: virt-145 (score:INFINITY) (role: Started) (id:cli-prefer-dummy0) > Constraint has been created, but resource WAS NOT moved. [root@virt-143 ~]# pcs resource move DummyBundle-0 virt-145 Error: DummyBundle-0 is not a valid resource [root@virt-143 ~]# crm_resource --resource DummyBundle-0 --move --node virt-145 Error performing operation: Update does not conform to the configured schema > Neither pcs not crm_resource is able to target the DummyBundle-0 pseudo resource which is expected. [root@virt-143 ~]# pcs resource move DummyBundle virt-145 Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource DummyBundle --move --node virt-145 Resource 'DummyBundle' not moved: active on multiple nodes Error performing operation: Invalid argument > Just like in the previous case the crm_resource gives us a misinformation about the number of locations being active. There should only be one active due to replicas=1. dummy1 is a clone with clone-max=1 ---------------------------------- [root@virt-143 ~]# pcs resource move dummy1 virt-145 Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1 --move --node virt-145 [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Enabled on: virt-145 (score:INFINITY) (role: Started) (id:cli-prefer-dummy1) > A clone with single instance was refused to move by pcs, but actually passed and WAS moved with crm_resource. There might be a room for improvement in pcs. [root@virt-143 ~]# pcs resource move dummy1-clone virt-146 Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1-clone --move --node virt-146 [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1-clone Enabled on: virt-146 (score:INFINITY) (role: Started) (id:cli-prefer-dummy1-clone) > When targeting clone ID, once again a clone with single instance was refused to move by pcs, but passed and WAS moved with crm_resource. dummy1 is a clone with clone-max=2 ---------------------------------- [root@virt-143 ~]# pcs resource move dummy1 virt-145 Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1 --move --node virt-145 [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Enabled on: virt-145 (score:INFINITY) (role: Started) (id:cli-prefer-dummy1) > With more than one clone instance crm_resource does perform a constraint-based move by moving ONE of the instance. [root@virt-143 ~]# pcs resource move dummy1-clone virt-146 Error: cannot move cloned resources [root@virt-143 ~]# [root@virt-143 ~]# crm_resource --resource dummy1-clone --move --node virt-146 [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1-clone Enabled on: virt-146 (score:INFINITY) (role: Started) (id:cli-prefer-dummy1-clone) > Here with clone ID and two instances crm_resource won't do us a favor anymore. Pcs keeps printing the same error as before. The conclusion for 'resource move' with node specification is pretty much aligned with the previous one: - pcs is not really up in sync with crm_resource behavior in terms of being able to move a single clone instance - support for moving a bundle in crm_resource is not on par with being able to move a clone, at least in case of just one instance being present due to: - 2 active locations being improperly reported - the move constraint of a primitive is incorrectly targeted to a non-existent DummyBundle-0 node. Performing tests of ban with crm_resource is pretty much identical to simple move (with no node specification), so I am going to mention pcs differences only: dummy0 is a resource inside DummyBundle bundle with replicas=1 -------------------------------------------------------------- [root@virt-143 ~]# pcs resource ban DummyBundle Error: error moving/banning/clearing resource Resource 'DummyBundle' not moved: active in 2 locations. You can prevent 'DummyBundle' from running on a specific location with: --ban --node <name> Error performing operation: Invalid argument [root@virt-143 ~]# crm_resource --resource DummyBundle --ban Resource 'DummyBundle' not moved: active in 2 locations. You can prevent 'DummyBundle' from running on a specific location with: --ban --node <name> Error performing operation: Invalid argument > The difference is in pcs output when targetting DummyBundle which no longer complains about a clone but instead passes the crm_resource message as is. We'll need a more user friendly message in pcs. dummy1 is a clone with clone-max=1 ---------------------------------- [root@virt-143 ~]# pcs resource ban dummy1 Warning: Creating location constraint cli-ban-dummy1-on-virt-143 with a score of -INFINITY for resource dummy1 on node virt-143. This will prevent dummy1 from running on virt-143 until the constraint is removed. This will be the case even if virt-143 is the last node in the cluster. [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-143 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-143) [root@virt-143 ~]# crm_resource --resource dummy1 --ban WARNING: Creating rsc_location constraint 'cli-ban-dummy1-on-virt-144' with a score of -INFINITY for resource dummy1 on virt-144. This will prevent dummy1 from running on virt-144 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-144 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-144 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-144) > The resource WAS moved in both cases so pcs and crm_resource are in sync regarding behavior. [root@virt-143 ~]# pcs resource ban dummy1-clone Warning: Creating location constraint cli-ban-dummy1-clone-on-virt-143 with a score of -INFINITY for resource dummy1-clone on node virt-143. This will prevent dummy1-clone from running on virt-143 until the constraint is removed. This will be the case even if virt-143 is the last node in the cluster. [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1-clone Disabled on: virt-143 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-clone-on-virt-143) [root@virt-143 ~]# crm_resource --resource dummy1-clone --ban WARNING: Creating rsc_location constraint 'cli-ban-dummy1-clone-on-virt-144' with a score of -INFINITY for resource dummy1-clone on virt-144. This will prevent dummy1-clone from running on virt-144 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-144 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1-clone Disabled on: virt-144 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-clone-on-virt-144) > Again the resource WAS moved in both cases so pcs and crm_resource are in sync regarding behavior. dummy1 is a clone with clone-max=2 ---------------------------------- [root@virt-143 ~]# pcs resource ban dummy1 Warning: Creating location constraint cli-ban-dummy1-on-virt-145 with a score of -INFINITY for resource dummy1 on node virt-145. This will prevent dummy1 from running on virt-145 until the constraint is removed. This will be the case even if virt-145 is the last node in the cluster. [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-145 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-145) [root@virt-143 ~]# crm_resource --resource dummy1 --ban WARNING: Creating rsc_location constraint 'cli-ban-dummy1-on-virt-143' with a score of -INFINITY for resource dummy1 on virt-143. This will prevent dummy1 from running on virt-143 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-143 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-143 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-143) > One of the resource instances WAS moved in both cases so pcs and crm_resource are in sync regarding behavior. [root@virt-143 ~]# pcs resource ban dummy1-clone Error: error moving/banning/clearing resource Resource 'dummy1-clone' not moved: active in 2 locations. You can prevent 'dummy1-clone' from running on a specific location with: --ban --node <name> Error performing operation: Invalid argument [root@virt-143 ~]# [root@virt-143 ~]# crm_resource --resource dummy1-clone --ban Resource 'dummy1-clone' not moved: active in 2 locations. You can prevent 'dummy1-clone' from running on a specific location with: --ban --node <name> Error performing operation: Invalid argument > The difference is in pcs output when targetting DummyBundle which no longer complains about a clone but instead passes the crm_resource message as is. We'll need a more user friendly message in pcs. The conclusion for 'resource ban' is the same for crm_resource, things are only different for pcs: - pcs now doesn't scream about clones anymore but instead pass crm_resource's error message directly, which we might need to modify a little - support for banning a bundle in crm_resource is not on par with being able to move a clone, at least in case of just one instance being present due to: - 2 active locations being improperly reported - the move constraint of a primitive is incorrectly targeted to a non-existent DummyBundle-0 node