Hide Forgot
Description of problem: In OSP7, under an updated RHEL 7.2, if you try to disable redis using this command: [heat-admin@overcloud-controller-0 ~]$ sudo pcs resource disable redis [heat-admin@overcloud-controller-0 ~]$ sudo crm_resource -V --wait you wait forever seeing something like this: error: clone_update_actions_interleave: Internal error: No action found for demote in redis:1 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:2 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:1 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:2 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:1 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:2 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:1 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:2 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:1 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:2 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:1 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:2 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:1 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:2 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:1 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:2 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:1 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:2 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:1 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:2 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:1 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:2 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:1 (then) error: clone_update_actions_interleave: Internal error: No action found for demote in redis:2 (then) The resource have constraints that should make the disable process straight. A similar problem related to redis happen if you do this test: 1) Stop galera 2) Stop rabbit 3) Stop redis 4) Start galera 5) Start rabbit 6) Start redis If you use wait on each operation, everything is fine until you try to start rabbit (step 5), then you wait forever (or the timeout you passed to pcs resource enable rabbitmq --wait <TIMEOUT>). If the sequence is instead this one: 1) Stop galera 2) Stop rabbit 3) Stop redis 4) Start galera 5) Start redis 6) Start rabbit Then everything ends fine. Note that the version of Pacemaker used was the one that fixed these two bugs: https://bugzilla.redhat.com/show_bug.cgi?id=1347806 https://bugzilla.redhat.com/show_bug.cgi?id=1349493 Version-Release number of selected component (if applicable): pacemaker-1.1.13-10.el7_2.4 How reproducible: Always
If you apply the following patch: diff --git a/tools/crm_resource_runtime.c b/tools/crm_resource_runtime.c index 42e8b07..bac9907 100644 --- a/tools/crm_resource_runtime.c +++ b/tools/crm_resource_runtime.c @@ -1360,7 +1360,9 @@ actions_are_pending(GListPtr actions) GListPtr action; for (action = actions; action != NULL; action = action->next) { - if (action_is_pending((action_t *) action->data)) { + action_t *a = (action_t *)action->data; + if (action_is_pending(a)) { + printf("Found: %s 0x%x\n", a->uuid, a->flags); return TRUE; } } and run: CIB_file=sosreport-overcloud-controller-0.localdomain-20161007091508/sos_commands/cluster/crm_report/overcloud-controller-0.localdomain/pe-input-45.bz2 tools/crm_resource --wait -VV against the http://file.rdu.redhat.com/~rscarazz/BZ1382068/pacemaker-1.1.15-11.el7/ tarball, then you'll see: Found: openstack-heat-api-cfn:2_monitor_60000 0x112 Found: openstack-heat-api-cfn:2_monitor_60000 0x112 ... Which indicates that although that openstack-heat-api-cfn:2_start_0 operation is correctly removed from the graph (because its dependancies depend on redis which is not allowed to start), we are still under the impression that the monitor op can run. This confuses the --wait logic
This is unlikely to be addressed in the 7.4 timeframe
Due to time constraints, this will not make 7.5
Moving to RHEL 8 since this will not be fixed in the 7.9 time frame
This issue is still present in the latest upstream pacemaker and is still a high priority. However new policy prevents us from keeping this report open, so an upstream bug has been filed for the issue, and this report will be closed. It will be reopened when developer time becomes available to address it.
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release. Therefore, it is being closed. If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.
This issue is still a high priority, and when developer time becomes available for it, we will reopen this bz.