Bug 1215976

Summary: Every few hours an Openstack controller is fenced on my HA cluster
Product: Red Hat Enterprise Linux 7 Reporter: Tzach Shefi <tshefi>
Component: pacemakerAssignee: Andrew Beekhof <abeekhof>
Status: CLOSED DUPLICATE QA Contact: cluster-qe <cluster-qe>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 7.1CC: cluster-maint, dvossel, jreznik, oblaut
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-15 22:48:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
gz of messages from all three nodes
none
crm_report log none

Description Tzach Shefi 2015-04-28 09:22:41 UTC 
Created attachment 1019589 [details]
gz of messages from all three nodes

Description of problem: Running RHOS6 A2 HA deployment, 3 controllers on pacemaker cluster + 2 compute nodes. Every few hours one of the compute cluster nodes is fenced. I also see another problem fencing caused shutdown rather than reboot bz1194301.  


Version-Release number of selected component (if applicable):
RHEL7.1 
Openstack HA RHOS6 A2
pacemaker-cli-1.1.12-22.el7_1.1.x86_64
pacemaker-1.1.12-22.el7_1.1.x86_64
pacemaker-cluster-libs-1.1.12-22.el7_1.1.x86_64
pacemaker-libs-1.1.12-22.el7_1.1.x86_6

How reproducible:
Happens every few hours. 

Steps to Reproduce:
1. Installed RHOS HA deployment with staypuft, which configured cluster stuff.
2.
3.

Actual results:
Cluster  nodes are fenced every few hours. 

Expected results:
Nodes shouldn't be fenced so often :( 

Additional info:
Attaching /var/log/messages from each node plus plus crm_report
Comment 1 Tzach Shefi 2015-04-28 09:23:25 UTC 
Created attachment 1019590 [details]
crm_report log
Comment 3 David Vossel 2015-04-28 14:10:38 UTC 
This is the libqb bug that is in the process of being zstreamed.

https://bugzilla.redhat.com/show_bug.cgi?id=1212297
Comment 4 Tzach Shefi 2015-05-04 07:26:51 UTC 
FYI after upgraded to libqb-0.17.1-1.el7_1.2 on my controller nodes, tip from above bz, fencing problem hasn't accorded in the past 24h.
Comment 5 Andrew Beekhof 2015-07-15 22:48:53 UTC 

*** This bug has been marked as a duplicate of bug 1212297 ***