Bug 1972273

Summary: pacemaker seems to end up in an unfence loop [rhel-8.4.0.z]
Product: Red Hat Enterprise Linux 8 Reporter: RHEL Program Management Team <pgm-rhel-tools>
Component: pacemakerAssignee: Ken Gaillot <kgaillot>
Status: CLOSED ERRATA QA Contact: cluster-qe <cluster-qe>
Severity: high Docs Contact:
Priority: high    
Version: 8.4CC: cfeist, cluster-maint, dabarzil, dciabrin, jeckersb, jmarcian, kwenning, lmiccini, sbradley
Target Milestone: betaKeywords: Triaged, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pacemaker-2.0.5-9.el8_4.2 Doc Type: Bug Fix
Doc Text:
Cause: Pacemaker's controller on a node might be elected the Designated Controller (DC) before its attribute manager learned an already-active remote node is remote. Consequence: The node's scheduler would not see any of the remote node's node attributes. If the cluster used unfencing, this could result in an unfencing loop. Fix: The attribute manager can now learn a remote node is remote via additional events, including the initial attribute sync at start-up. Result: No unfencing loop occurs, regardless of which node is elected DC.
Story Points: ---
Clone Of: 1961857 Environment:
Last Closed: 2021-09-21 08:39:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1961857    
Bug Blocks:    

Comment 1 Ken Gaillot 2021-06-22 16:45:15 UTC
Fixed upstream as of commit 540d7413

Comment 3 Julia Marciano 2021-08-02 13:19:55 UTC
Verified:
After the fixed (pacemaker-2.0.5-9.el8_4.2)  packages were installed on controller and compute nodes, and after executing all issue-reproducing steps (please see [Steps] below), it seems that the issue doesn't reproduce anymore. No spurious, node-unfenced related messages were found in the pacemaker log:
(undercloud) [stack@undercloud-0 ~]$ for i in $(openstack server list -c Networks -f value | cut -f2 -d\=); do  ssh $i "hostname; rpm -q pacemaker; sudo grep -e 'Cannot update #node-unfenced' /var/log/pacemaker/pacemaker.log"; done 2>/dev/null
controller-1
pacemaker-2.0.5-9.el8_4.2.x86_64
controller-0
pacemaker-2.0.5-9.el8_4.2.x86_64
controller-2
pacemaker-2.0.5-9.el8_4.2.x86_64
compute-1
pacemaker-2.0.5-9.el8_4.2.x86_64
compute-0
pacemaker-2.0.5-9.el8_4.2.x86_64

[Steps]:
1. cont-1# sudo shutdown -h now   // curent-DC was cont-0
2. comp-0# sudo reboot
3. openstack baremetal node power on <cont-1 UUID>
4.  Wait for cluster to include cont-1
5. cont-0# echo b > /proc/sysrq-trigger  --> cont-2 became a DC
6. cont-2# echo b > /proc/sysrq-trigger -->  cont-1 became a DC

Comment 14 errata-xmlrpc 2021-09-21 08:39:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (pacemaker bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3578