Description of problem: nfs-ganesha disable fails to bring down ganehsa on all nodes. This command I have tried earlier also, but this time the only difference is that I have selinux enforced and I tried to execute nfs-ganesha disable from some other node rather from the same node where nfs-ganesha enable command was executed Version-Release number of selected component (if applicable): glusterfs-3.7.0-3.el6rhs.x86_64 nfs-ganesha-2.2.0-2.el6rhs.x86_64 How reproducible: nfs-disable failure is intermittent. nfs-disable in the scenario mentioned above is executed first time Steps to Reproduce: have a setup of 4 nodes say, nfs9,nfs10,nfs11,nfs12 1a. have selinux enforced on all nodes 1. from nfs9 using nfs-ganesha enable command bring up nfs-ganesha, after completing pre-requisites 2. from nfs10 teardown nfs-ganesha Actual results: step 2 result, [root@nfs10 ~]# gluster nfs-ganesha disable Error : Request timed out selinux update from nfs10, type=CRYPTO_KEY_USER msg=audit(1433868259.369:1564): user pid=30389 uid=0 auid=0 ses=91 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=36:ca:3b:7d:fd:b4:80:c8:ca:b5:2f:2c:46:4f:be:53 direction=? spid=30389 suid=0 exe="/usr/sbin/sshd" hostname=? addr=10.70.47.127 terminal=? res=success' type=CRYPTO_KEY_USER msg=audit(1433868259.370:1565): user pid=30389 uid=0 auid=0 ses=91 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=ec:96:29:c1:21:56:39:0b:3f:e9:6d:d9:07:8f:15:f4 direction=? spid=30389 suid=0 exe="/usr/sbin/sshd" hostname=? addr=10.70.47.127 terminal=? res=success' type=CRED_REFR msg=audit(1433868259.371:1566): user pid=30389 uid=0 auid=0 ses=91 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/sshd" hostname=nfs9 addr=10.70.47.127 terminal=ssh res=success' type=USER_END msg=audit(1433868259.533:1567): user pid=30386 uid=0 auid=0 ses=91 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/sshd" hostname=nfs9 addr=10.70.47.127 terminal=ssh res=success' type=CRED_DISP msg=audit(1433868259.533:1568): user pid=30386 uid=0 auid=0 ses=91 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/sshd" hostname=nfs9 addr=10.70.47.127 terminal=ssh res=success' type=USER_END msg=audit(1433868259.534:1569): user pid=30386 uid=0 auid=0 ses=91 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=nfs9 addr=10.70.47.127 terminal=ssh res=success' type=USER_LOGOUT msg=audit(1433868259.534:1570): user pid=30386 uid=0 auid=0 ses=91 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=login id=0 exe="/usr/sbin/sshd" hostname=nfs9 addr=10.70.47.127 terminal=ssh res=success' type=CRYPTO_KEY_USER msg=audit(1433868259.534:1571): user pid=30386 uid=0 auid=0 ses=91 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=36:ca:3b:7d:fd:b4:80:c8:ca:b5:2f:2c:46:4f:be:53 direction=? spid=30386 suid=0 exe="/usr/sbin/sshd" hostname=? addr=10.70.47.127 terminal=? res=success' type=CRYPTO_KEY_USER msg=audit(1433868259.534:1572): user pid=30386 uid=0 auid=0 ses=91 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=server fp=ec:96:29:c1:21:56:39:0b:3f:e9:6d:d9:07:8f:15:f4 direction=? spid=30386 suid=0 exe="/usr/sbin/sshd" hostname=? addr=10.70.47.127 terminal=? res=success' type=CRYPTO_KEY_USER msg=audit(1433868259.534:1573): user pid=30386 uid=0 auid=0 ses=91 subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='op=destroy kind=session fp=? direction=both spid=30386 suid=0 rport=43189 laddr=10.70.47.130 lport=22 exe="/usr/sbin/sshd" hostname=? addr=10.70.47.127 terminal=? res=success' (END) Here if you see it is mentioning about IP 10.70.47.127 and it is the IP of nfs9 nfs10 /var/log/messages, Jun 9 22:22:11 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:22:21 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:22:31 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:22:41 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:22:51 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:23:01 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:23:11 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:23:21 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:23:31 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:23:41 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:23:51 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:24:01 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:24:11 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:24:21 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:24:31 nfs10 fenced[5100]: cpg_leave error retrying Jun 9 22:24:41 nfs10 fenced[5100]: cpg_leave error retrying ganesha process on all nodes post timeout of the command nfs-ganesha disable, nfs9 root 10642 1 0 20:42 ? 00:00:00 /usr/bin/ganesha.nfsd -L /var/log/ganesha.log -f /etc/ganesha/ganesha.conf -N NIV_EVENT -p /var/run/ganesha.nfsd.pid root 30910 1671 0 21:28 ? 00:00:00 sh /usr/libexec/ganesha/ganesha-ha.sh teardown /etc/ganesha -------- nfs10 root 30389 30386 2 22:14 ? 00:00:00 bash -c ps -eaf | grep ganesha* root 30399 30389 0 22:14 ? 00:00:00 grep ganesha* -------- nfs11 root 28653 28650 2 00:59 ? 00:00:00 bash -c ps -eaf | grep ganesha* root 28663 28653 0 00:59 ? 00:00:00 grep ganesha* -------- nfs12 root 28449 28446 0 00:59 ? 00:00:00 bash -c ps -eaf | grep ganesha* root 28459 28449 0 00:59 ? 00:00:00 grep ganesha* Expected results: disable should teardown the nfs-ganesha cluster properly. Additional info:
Created attachment 1036736 [details] sosreport of nfs9
Request Milos to confirm if this issue happening because of selinux?
Created attachment 1036770 [details] sosreport of nfs10
I saw following USER_AVCs in the attachment: type=USER_AVC msg=audit(1433848674.760:1142): user pid=1672 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { send_msg } for msgtype=method_return dest=:1.9 spid=3622 tpid=19488 scontext=unconfined_u:system_r:glusterd_t:s0 tcontext=unconfined_u:system_r:cluster_t:s0 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1433862743.548:1359): user pid=1667 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=4850 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' You can look at USER_AVC as a SELinux denial in user-space. These USER_AVCs are related to D-bus communication (tclass=dbus). When D-bus communication is blocked by SELinux it usually leads to timeouts. Short answer: yes. The timeout happens because of SELinux.
Updated the setup based on recommendations given in, https://bugzilla.redhat.com/show_bug.cgi?id=1220999#c9 Executed the gluster nfs-ganesha disable command and it again timed out, here is the result of the avc logs from the participating nodes, nfs9 type=USER_AVC msg=audit(1433782920.072:865): user pid=1594 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=3905 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1433846400.563:1091): user pid=1687 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=4154 scontext=unconfined_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1433862731.217:1445): user pid=1690 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=10642 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' -------- nfs10 type=USER_AVC msg=audit(1433782931.883:790): user pid=1593 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=3588 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1433846411.804:1096): user pid=1672 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=3622 scontext=unconfined_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1433848674.760:1142): user pid=1672 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { send_msg } for msgtype=method_return dest=:1.9 spid=3622 tpid=19488 scontext=unconfined_u:system_r:glusterd_t:s0 tcontext=unconfined_u:system_r:cluster_t:s0 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1433862743.548:1359): user pid=1667 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=4850 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' -------- nfs11 type=USER_AVC msg=audit(1433792817.393:790): user pid=1586 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=3568 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1433856297.894:1090): user pid=1689 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=3600 scontext=unconfined_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1433858560.953:1148): user pid=1689 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { send_msg } for msgtype=method_return dest=:1.9 spid=3600 tpid=20120 scontext=unconfined_u:system_r:glusterd_t:s0 tcontext=unconfined_u:system_r:cluster_t:s0 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1433872627.906:1359): user pid=1680 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=4377 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' -------- nfs12 type=USER_AVC msg=audit(1433792816.850:783): user pid=1591 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=3559 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1433856297.202:897): user pid=1669 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=3285 scontext=unconfined_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1433872627.792:1359): user pid=1679 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { acquire_svc } for service=org.ganesha.nfsd spid=4617 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus exe="/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' -------- Milos let me know if more information is required for triaging the problem.
For triaging purposes it's enough. I will update the policy module in BZ#1220999 to overcome these problems.
I updated the setup as per, https://bugzilla.redhat.com/show_bug.cgi?id=1220999#c11 and tried to execute these two steps, 1. bring up ganesha using command "gluster nfs-ganesha enable"--- Pass 2. bring down ganesha using command "gluster nfs-ganesha disable" --- fails, basically time out problem seen. /var/log/audit/audit.log, type=USER_ACCT msg=audit(1433949001.072:5099): user pid=27309 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_ACQ msg=audit(1433949001.073:5100): user pid=27309 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=LOGIN msg=audit(1433949001.087:5101): pid=27309 uid=0 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 old auid=4294967295 new auid=0 old ses=4294967295 new ses=350 type=USER_START msg=audit(1433949001.093:5102): user pid=27309 uid=0 auid=0 ses=350 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_DISP msg=audit(1433949001.191:5103): user pid=27309 uid=0 auid=0 ses=350 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_END msg=audit(1433949001.191:5104): user pid=27309 uid=0 auid=0 ses=350 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_ACCT msg=audit(1433949601.222:5105): user pid=31195 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_ACQ msg=audit(1433949601.222:5106): user pid=31195 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=LOGIN msg=audit(1433949601.223:5107): pid=31195 uid=0 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 old auid=4294967295 new auid=0 old ses=4294967295 new ses=351 type=USER_START msg=audit(1433949601.227:5108): user pid=31195 uid=0 auid=0 ses=351 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_DISP msg=audit(1433949601.298:5109): user pid=31195 uid=0 auid=0 ses=351 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_END msg=audit(1433949601.298:5110): user pid=31195 uid=0 auid=0 ses=351 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_ACCT msg=audit(1433950201.308:5111): user pid=3180 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_ACQ msg=audit(1433950201.308:5112): user pid=3180 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=LOGIN msg=audit(1433950201.308:5113): pid=3180 uid=0 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 old auid=4294967295 new auid=0 old ses=4294967295 new ses=352 type=USER_START msg=audit(1433950201.311:5114): user pid=3180 uid=0 auid=0 ses=352 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_DISP msg=audit(1433950201.395:5115): user pid=3180 uid=0 auid=0 ses=352 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_END msg=audit(1433950201.395:5116): user pid=3180 uid=0 auid=0 ses=352 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_ACCT msg=audit(1433950261.401:5117): user pid=3626 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_ACQ msg=audit(1433950261.401:5118): user pid=3626 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=LOGIN msg=audit(1433950261.402:5119): pid=3626 uid=0 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 old auid=4294967295 new auid=0 old ses=4294967295 new ses=353 type=USER_START msg=audit(1433950261.406:5120): user pid=3626 uid=0 auid=0 ses=353 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_DISP msg=audit(1433950261.529:5121): user pid=3626 uid=0 auid=0 ses=353 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_END msg=audit(1433950261.529:5122): user pid=3626 uid=0 auid=0 ses=353 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_ACCT msg=audit(1433950801.549:5123): user pid=7529 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_ACQ msg=audit(1433950801.549:5124): user pid=7529 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=LOGIN msg=audit(1433950801.550:5125): pid=7529 uid=0 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 old auid=4294967295 new auid=0 old ses=4294967295 new ses=354 type=USER_START msg=audit(1433950801.556:5126): user pid=7529 uid=0 auid=0 ses=354 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_DISP msg=audit(1433950801.632:5127): user pid=7529 uid=0 auid=0 ses=354 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_END msg=audit(1433950801.632:5128): user pid=7529 uid=0 auid=0 ses=354 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=AVC msg=audit(1433951106.046:5129): avc: denied { execute } for pid=9360 comm="env" name="pacemaker" dev=dm-0 ino=661749 scontext=unconfined_u:system_r:glusterd_t:s0 tcontext=system_u:object_r:cluster_initrc_exec_t:s0 tclass=file type=AVC msg=audit(1433951106.046:5129): avc: denied { execute_no_trans } for pid=9360 comm="env" path="/etc/rc.d/init.d/pacemaker" dev=dm-0 ino=661749 scontext=unconfined_u:system_r:glusterd_t:s0 tcontext=system_u:object_r:cluster_initrc_exec_t:s0 tclass=file type=SYSCALL msg=audit(1433951106.046:5129): arch=c000003e syscall=59 success=yes exit=0 a0=7ffe9678af3d a1=7ffe96789fe8 a2=12b4030 a3=fffffff8 items=0 ppid=9353 pid=9360 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=125 comm="pacemaker" exe="/bin/bash" subj=unconfined_u:system_r:glusterd_t:s0 key=(null) type=AVC msg=audit(1433951110.080:5130): avc: denied { create } for pid=9580 comm="sed" name="sed5JLPhF" scontext=unconfined_u:system_r:glusterd_t:s0 tcontext=system_u:object_r:etc_t:s0 tclass=file type=SYSCALL msg=audit(1433951110.080:5130): arch=c000003e syscall=2 success=yes exit=4 a0=756ee0 a1=c2 a2=180 a3=0 items=0 ppid=9572 pid=9580 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=125 comm="sed" exe="/bin/sed" subj=unconfined_u:system_r:glusterd_t:s0 key=(null) type=USER_ACCT msg=audit(1433951401.681:5131): user pid=9589 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_ACQ msg=audit(1433951401.681:5132): user pid=9589 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=LOGIN msg=audit(1433951401.693:5133): pid=9589 uid=0 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 old auid=4294967295 new auid=0 old ses=4294967295 new ses=355 type=USER_START msg=audit(1433951401.699:5134): user pid=9589 uid=0 auid=0 ses=355 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_DISP msg=audit(1433951401.800:5135): user pid=9589 uid=0 auid=0 ses=355 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_END msg=audit(1433951401.802:5136): user pid=9589 uid=0 auid=0 ses=355 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_ACCT msg=audit(1433952001.818:5137): user pid=9610 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_ACQ msg=audit(1433952001.819:5138): user pid=9610 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=LOGIN msg=audit(1433952001.819:5139): pid=9610 uid=0 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 old auid=4294967295 new auid=0 old ses=4294967295 new ses=356 type=USER_START msg=audit(1433952001.824:5140): user pid=9610 uid=0 auid=0 ses=356 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_DISP msg=audit(1433952001.895:5141): user pid=9610 uid=0 auid=0 ses=356 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_END msg=audit(1433952001.895:5142): user pid=9610 uid=0 auid=0 ses=356 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
It seems that gluster daemon also wants to run some cluster services. The AVCs contain at least pacemaker and its init script. I will update the policy module in BZ#1220999 to overcome these problems.
[root@nfs11 ~]# gluster nfs-ganesha disable This will take a few minutes to complete. Please wait .. nfs-ganesha : success works fine and hence moving this BZ to verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-1495.html