Bug 1170367

Summary: rubygem-staypuft: [nonHA] Puppet error: Stage[main]/Quickstack::Pacemaker::Galera/Exec[galera-online]/returns change from notrun to 0 failed: /usr/bin/clustercheck >/dev/null returned 1 instead of one of [0] - the deployment gets paused with error.
Product: Red Hat OpenStack Reporter: Alexander Chuzhoy <sasha>
Component: openstack-selinuxAssignee: Ryan Hallisey <rhallise>
Status: CLOSED ERRATA QA Contact: Alexander Chuzhoy <sasha>
Severity: urgent Docs Contact:
Priority: urgent    
Version: unspecifiedCC: cwolfe, dmacpher, lhh, mburns, mgrepl, rhallise, sclewis, yeylon
Target Milestone: ga   
Target Release: Installer   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-selinux-0.6.3-1.el7ost Doc Type: Bug Fix
Doc Text:
SELinux prevented a successful Galera configuration during deployment due to a disabled boolean (daemons_enable_cluster_mode). This fix enables the boolean, which allows Galera to configure successfully.
Story Points: ---
Clone Of:
: 1171827 (view as bug list) Environment:
Last Closed: 2015-02-09 15:16:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1171827    
Attachments:
Description Flags
/var/log/messages file from the controller
none
contains: audit.log mariadb.log messages pacemaker.log none

Description Alexander Chuzhoy 2014-12-03 22:14:37 UTC
rubygem-staypuft: [nonHA] Puppet error: Stage[main]/Quickstack::Pacemaker::Galera/Exec[galera-online]/returns	change from notrun to 0 failed: /usr/bin/clustercheck >/dev/null returned 1 instead of one of [0] - the deployment gets paused with error.


Environment:
openstack-foreman-installer-3.0.3-1.el7ost.noarch
ruby193-rubygem-staypuft-0.5.1-1.el7ost.noarch
ruby193-rubygem-foreman_openstack_simplify-0.0.6-8.el7ost.noarch
rhel-osp-installer-client-0.5.1-1.el7ost.noarch
openstack-puppet-modules-2014.2.6-1.el7ost.noarch
rhel-osp-installer-0.5.1-1.el7ost.noarch


Steps to reproduce:
1. Install rhel-osp-installer.
2. Create/run a deployment with a single controller + 1 compute.

Result:
The deployment gets paused with error.
Analyzing the reports I see this puppet error: Stage[main]/Quickstack::Pacemaker::Galera/Exec[galera-online]/returns	change from notrun to 0 failed: /usr/bin/clustercheck >/dev/null returned 1 instead of one of [0] - the deployment gets paused with error.

Expected result:
This puppet error shouldn't occur.

Comment 1 Alexander Chuzhoy 2014-12-03 22:16:57 UTC
Created attachment 964361 [details]
/var/log/messages file from the controller

Comment 2 Crag Wolfe 2014-12-03 22:53:40 UTC
selinux is preventing pacemaker from starting galera.  E.g., starting the pacemaker galera resource agent looked good at first:

 Master/Slave Set: galera-master [galera]
     Slaves: [ pcmk-maca25400702877 ]

but it never got promoted to Master.  After setting selinux to permissive, pacemaker was able to start galera.

The root cause looks like (from /var/log/audit/audit.log):
type=AVC msg=audit(1417646108.342:337): avc:  denied  { read } for  pid=18968 comm="mysqld_safe" name="cores" dev="dm-0" ino=51125914 scontext=system_u:sys
m_r:mysqld_safe_t:s0 tcontext=system_u:object_r:cluster_var_lib_t:s0 tclass=dir

Comment 3 Crag Wolfe 2014-12-03 23:03:25 UTC
Created attachment 964366 [details]
contains: audit.log  mariadb.log  messages  pacemaker.log

audit.log includes the mysql-related AVC.  Other pertinent log files included.

Comment 4 Ryan Hallisey 2014-12-03 23:21:42 UTC
type=AVC msg=audit(1417638763.636:183): avc: denied { read } for pid=19681 comm="mysqld_safe" name="cores" dev="dm-0" ino=51125914 scontext=system_u:system_r:mysqld_safe_t:s0 tcontext=system_u:object_r:cluster_var_lib_t:s0 tclass=dir

$ setsebool daemons_enable_cluster_mode 1

Will review with Lon and Miroslav.

Comment 5 Miroslav Grepl 2014-12-08 10:20:23 UTC
Yes, we have

$ setsebool daemons_enable_cluster_mode 1

for these cases.

Comment 7 Alexander Chuzhoy 2014-12-08 23:40:28 UTC
Verified:
Environment:
rhel-osp-installer-client-0.5.1-1.el7ost.noarch
openstack-puppet-modules-2014.2.6-1.el7ost.noarch
openstack-foreman-installer-3.0.5-1.el7ost.noarch
rhel-osp-installer-0.5.1-1.el7ost.noarch
ruby193-rubygem-staypuft-0.5.3-1.el7ost.noarch
ruby193-rubygem-foreman_openstack_simplify-0.0.6-8.el7ost.noarch

The reported issue doesn't re-produce.

Comment 9 errata-xmlrpc 2015-02-09 15:16:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0156.html