Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2046446

Summary: Removing pacemaker_remote node ends up in fence action on the remote node
Product: Red Hat Enterprise Linux 8 Reporter: Ken Gaillot <kgaillot>
Component: pacemakerAssignee: Ken Gaillot <kgaillot>
Status: CLOSED ERRATA QA Contact: cluster-qe <cluster-qe>
Severity: medium Docs Contact:
Priority: high    
Version: 8.6CC: cluster-maint, cluster-qe, iromanos, mmazoure, msmazova
Target Milestone: rcKeywords: Regression, Triaged
Target Release: 8.6Flags: pm-rhel: mirror+
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: pacemaker-2.1.2-4.el8 Doc Type: No Doc Update
Doc Text:
This issue was not in a released build
 Story Points: ---
Clone Of: 2039399 Environment:
Last Closed: 2022-05-10 14:10:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2039399    
Bug Blocks:    

Description Ken Gaillot 2022-01-26 17:17:02 UTC 
+++ This bug was initially created as a clone of Bug #2039399 +++

Description of problem:
When removing pacemaker_remote node from cluster configuration (via 'pcs cluster node remove-remote'), fence action occurs on the remote node.


Version-Release number of selected component (if applicable):
pacemaker-2.1.2-1.el9.x86_64


How reproducible:
always


Steps to Reproduce:
[root@virt-255 ~]# pcs status
Cluster name: STSRHTS22418
Cluster Summary:
  * Stack: corosync
  * Current DC: virt-256 (version 2.1.2-1.el9-ada5c3b36e2) - partition with quorum
  * Last updated: Tue Jan 11 15:29:56 2022
  * Last change:  Tue Jan 11 15:29:01 2022 by root via cibadmin on virt-255
  * 4 nodes configured
  * 4 resource instances configured

Node List:
  * Online: [ virt-255 virt-256 virt-261 virt-262 ]

Full List of Resources:
  * fence-virt-255	(stonith:fence_xvm):	 Started virt-255
  * fence-virt-256	(stonith:fence_xvm):	 Started virt-256
  * fence-virt-261	(stonith:fence_xvm):	 Started virt-261
  * fence-virt-262	(stonith:fence_xvm):	 Started virt-262

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

[root@virt-255 ~]# pcs cluster node remove virt-262
Destroying cluster on hosts: 'virt-262'...
virt-262: Successfully destroyed cluster
Sending updated corosync.conf to nodes...
virt-261: Succeeded
virt-255: Succeeded
virt-256: Succeeded
virt-255: Corosync configuration reloaded

[root@virt-255 ~]# pcs cluster node add-remote virt-262
No addresses specified for host 'virt-262', using 'virt-262'
Sending 'pacemaker authkey' to 'virt-262'
virt-262: successful distribution of the file 'pacemaker authkey'
Requesting 'pacemaker_remote enable', 'pacemaker_remote start' on 'virt-262'
virt-262: successful run of 'pacemaker_remote enable'
virt-262: successful run of 'pacemaker_remote start'

[root@virt-255 ~]# pcs status | grep "Node List" -A 2
Node List:
  * Online: [ virt-255 virt-256 virt-261 ]
  * RemoteOnline: [ virt-262 ]

## removing the remote node

[root@virt-255 ~]# pcs cluster node remove-remote virt-262
Requesting 'pacemaker_remote disable', 'pacemaker_remote stop' on 'virt-262'
virt-262: successful run of 'pacemaker_remote disable'
virt-262: successful run of 'pacemaker_remote stop'
Requesting remove 'pacemaker authkey' from 'virt-262'
virt-262: successful removal of the file 'pacemaker authkey'
Deleting Resource - virt-262
[root@virt-255 ~]# echo $?
0

> This will stuck on 'Deleting Resource - virt-262' line for a few minutes, fence of the remote node will happen after that.


Actual results:
[root@virt-255 ~]# pcs stonith history
reboot of virt-262 successful: delegate=virt-261, client=pacemaker-controld.439906, origin=virt-256, completed='1970-01-05 01:18:46 +01:00'
1 event found


Expected results:
No fence action will occur


Additional info:
The same issue is present also in pacemaker-2.1.2-2, both on RHEL8 and RHEL9.
The issue is not present in version pacemaker-2.1.0-11.el9 and lower.

<snip>

--- Additional comment from Ken Gaillot on 2022-01-26 17:15:59 UTC ---

Fixed upstream by commit 16928cfc69
Comment 5 Ilias Romanos 2022-02-24 14:56:43 UTC 
[root@virt-564 ~]# rpm -qa pacemaker
pacemaker-2.1.2-4.el8.x86_64

[root@virt-564 ~]# pcs status
Cluster name: STSRHTS542
Cluster Summary:
  * Stack: corosync
  * Current DC: virt-564 (version 2.1.2-4.el8-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Feb 24 15:50:20 2022
  * Last change:  Thu Feb 24 15:45:20 2022 by root via cibadmin on virt-564
  * 5 nodes configured
  * 5 resource instances configured

Node List:
  * Online: [ virt-564 virt-565 virt-569 virt-570 virt-571 ]

Full List of Resources:
  * fence-virt-564	(stonith:fence_xvm):	 Started virt-564
  * fence-virt-565	(stonith:fence_xvm):	 Started virt-565
  * fence-virt-569	(stonith:fence_xvm):	 Started virt-569
  * fence-virt-570	(stonith:fence_xvm):	 Started virt-570
  * fence-virt-571	(stonith:fence_xvm):	 Started virt-571

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

[root@virt-564 ~]# pcs cluster node remove virt-571
Destroying cluster on hosts: 'virt-571'...
virt-571: Successfully destroyed cluster
Sending updated corosync.conf to nodes...
virt-565: Succeeded
virt-569: Succeeded
virt-564: Succeeded
virt-570: Succeeded
virt-564: Corosync configuration reloaded

[root@virt-564 ~]# pcs cluster node add-remote virt-571
No addresses specified for host 'virt-571', using 'virt-571'
Sending 'pacemaker authkey' to 'virt-571'
virt-571: successful distribution of the file 'pacemaker authkey'
Requesting 'pacemaker_remote enable', 'pacemaker_remote start' on 'virt-571'
virt-571: successful run of 'pacemaker_remote enable'
virt-571: successful run of 'pacemaker_remote start'

[root@virt-564 ~]# pcs status | grep "Node List" -A 2
Node List:
  * Online: [ virt-564 virt-565 virt-569 virt-570 ]
  * RemoteOnline: [ virt-571 ]

[root@virt-564 ~]# pcs cluster node remove-remote virt-571
Requesting 'pacemaker_remote disable', 'pacemaker_remote stop' on 'virt-571'
virt-571: successful run of 'pacemaker_remote disable'
virt-571: successful run of 'pacemaker_remote stop'
Requesting remove 'pacemaker authkey' from 'virt-571'
virt-571: successful removal of the file 'pacemaker authkey'
Deleting Resource - virt-571

[root@virt-564 ~]# echo $?
0

[root@virt-564 ~]# sleep 120 && pcs stonith history
0 events found
Comment 7 errata-xmlrpc 2022-05-10 14:10:03 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (pacemaker bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1885