Bug 1171827

Summary: openstack-foreman-installer: Puppet error: Stage[main]/Quickstack::Pacemaker::Galera/Exec[galera-online]/returns change from notrun to 0 failed: /usr/bin/clustercheck >/dev/null returned 1 instead of one of [0] - the deployment gets paused with error.
Product: Red Hat OpenStack Reporter: Alexander Chuzhoy <sasha>
Component: openstack-foreman-installerAssignee: Jason Guiditta <jguiditt>
Status: CLOSED ERRATA QA Contact: Alexander Chuzhoy <sasha>
Severity: urgent Docs Contact:
Priority: urgent    
Version: unspecifiedCC: ajeain, cwolfe, lhh, mburns, mgrepl, morazi, rhallise, rhos-maint, sclewis, yeylon
Target Milestone: ga   
Target Release: Installer   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-selinux-0.6.3-1.el7ost Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1170367 Environment:
Last Closed: 2015-02-09 15:17:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1170367    
Bug Blocks: 1177026    
Attachments:
Description Flags
audit.log contains the denial jayg wrote in a comment 2 none

Description Alexander Chuzhoy 2014-12-08 17:02:29 UTC
+++ This bug was initially created as a clone of Bug #1170367 +++

rubygem-staypuft: [nonHA] Puppet error: Stage[main]/Quickstack::Pacemaker::Galera/Exec[galera-online]/returns	change from notrun to 0 failed: /usr/bin/clustercheck >/dev/null returned 1 instead of one of [0] - the deployment gets paused with error.


Environment:
openstack-foreman-installer-3.0.3-1.el7ost.noarch
ruby193-rubygem-staypuft-0.5.1-1.el7ost.noarch
ruby193-rubygem-foreman_openstack_simplify-0.0.6-8.el7ost.noarch
rhel-osp-installer-client-0.5.1-1.el7ost.noarch
openstack-puppet-modules-2014.2.6-1.el7ost.noarch
rhel-osp-installer-0.5.1-1.el7ost.noarch


Steps to reproduce:
1. Install rhel-osp-installer.
2. Create/run a deployment with a single controller + 1 compute.

Result:
The deployment gets paused with error.
Analyzing the reports I see this puppet error: Stage[main]/Quickstack::Pacemaker::Galera/Exec[galera-online]/returns	change from notrun to 0 failed: /usr/bin/clustercheck >/dev/null returned 1 instead of one of [0] - the deployment gets paused with error.

Apparently this reproduces with selinux set to 0.

Expected result:
This puppet error shouldn't occur.

Comment 1 Alexander Chuzhoy 2014-12-08 17:03:39 UTC
Reproduces with:
openstack-foreman-installer-3.0.4-1.el7ost.noarch
openstack-puppet-modules-2014.2.6-1.el7ost.noarch
rhel-osp-installer-client-0.5.1-1.el7ost.noarch
rhel-osp-installer-0.5.1-1.el7ost.noarch
ruby193-rubygem-staypuft-0.5.2-1.el7ost.noarch
ruby193-rubygem-foreman_openstack_simplify-0.0.6-8.el7ost.noarch

Comment 2 Jason Guiditta 2014-12-08 20:19:13 UTC
We saw on the control node, the following avc:

type=USER_AVC msg=audit(1418064153.323:204): pid=532 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  denied  { send_msg } for msgtype=method_return dest=:1.33 spid=531 tpid=19138 scontext=system_u:system_r:systemd_logind_t:s0 tcontext=system_u:system_r:cluster_t:s0 tclass=dbus  exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'

We set selinux to permissive an reran, seemed to work fine, though of course, the avc still appears in audit.log

Comment 3 Crag Wolfe 2014-12-08 20:26:59 UTC
Created attachment 966002 [details]
audit.log contains the denial jayg wrote in a comment 2

Comment 4 Ryan Hallisey 2014-12-08 21:43:11 UTC
There was a syntax error in openstack-selinux-0.6.1-2.el7ost that caused the module to break and not enforce any policy.

http://pastebin.test.redhat.com/251115

There is a rule being dontaudited here that is causing things to break and that's why we don't see any AVCs in the logs, but only a USER_AVC. 

Test again with openstack-selinux-0.6.3-1.el7ost and the syntax error will be gone so the problem should go away.

Comment 7 Alexander Chuzhoy 2015-01-16 22:23:08 UTC
Verified:
Environment:
ruby193-rubygem-staypuft-0.5.12-1.el7ost.noarch
openstack-puppet-modules-2014.2.8-1.el7ost.noarch
ruby193-rubygem-foreman_openstack_simplify-0.0.6-8.el7ost.noarch
openstack-foreman-installer-3.0.10-2.el7ost.noarch
rhel-osp-installer-0.5.5-1.el7ost.noarch
rhel-osp-installer-client-0.5.5-1.el7ost.noarch


The reported issue doesn't reproduce.

Comment 9 errata-xmlrpc 2015-02-09 15:17:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0156.html